Endocrine, Transcriptomic and Social Regulation of Division of Labor in Honey Bees

Home , Alloparenting, Diacamma ceylonense

ENDOCRINE, TRANSCRIPTOMIC AND SOCIAL REGULATION OF DIVISION OF LABOR IN HONEY BEES

ADAM R. HAMILTON

DISSERTATION

Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Neuroscience in the Graduate College of the University of Illinois at Urbana-Champaign, 2018

Urbana, Illinois

Doctoral Committee:

Professor Gene E. Robinson, Chair Professor Lisa J. Stubbs Professor Emeritus Rhanor Gillette Associate Professor Justin S. Rhodes

Abstract

Division of labor is a central facet of complex societies. Task specialization by individual members of the society (theoretically) increases the overall productivity and the fitness of the group. The work presented in this dissertation extends existing knowledge about the regulation of task-related behavioral states at the level of the individual and social group by using the honeybee as a model organism. Chapter 1 reviews the extensive literature regarding the contribution of endocrine signaling (including endocrine-mediated transcriptional cascades) to division of labor in the social Hymenoptera. It also presents a theoretical framework for the evolution of division of labor via the cooption and neofunctionalization of endocrine-mediated signaling and transcription and suggests future lines of research to investigate these phenomena. Chapter 2 investigates the transcriptomic architecture underlying two of the tasks associated with division of labor (broodcare and foraging) using a novel combination of RNA sequencing and informatic analyses. In addition to identifying a key set of transcription factors (TFs) as putative regulators of broodcare or foraging behavior, it presents findings that suggest that coherent modules of coregulated genes are critical for task-related behavioral states. It thereby extends our understanding of how division of labor might be regulated at the transcriptomic level. Chapter 3 probes the regulatory logic underlying this architecture by investigating whether connections between TFs and their targets are labile. Using both bioinformatic analyses and RNAi coupled to behavioral assays and endocrine treatments, it presents significant evidence that the TF-target connectivity can be rewired as a function of behavioral state, social context and neuroendocrine state. This demonstrates how behavioral plasticity related to division of labor can arise at the transcriptomic level. Finally, Chapter 4 links division of labor to social networks involving trophallaxis (exchange of oral secretions and food). It shows that not only are task-related behaviors associated with differences in social interactivity, but that group-level social properties can be altered by hormone treatments that shift division of labor. Chapter 4 also demonstrates that certain emergent properties (such as information flow) are unaffected by such treatments and may represent core features of trophallactic communication in bees. Therefore, the findings presented in this chapter represent an important first step toward deciphering the role of direct communication in mediating division of labor.

Dedicated to my son, Arthur

iii

Acknowledgements

Science is fundamentally a collaborative enterprise. My friend and mentor Rhanor Gillette often comments that some of the most groundbreaking science begins with a simple conversation over cocktail napkins. In that sense, more people have contributed to the work in this dissertation than I could ever hope to acknowledge directly. Nonetheless, they will always have my thanks.

There are, however, some individuals I would like to recognize explicitly: my friends (including my fellow lab members and my lab manager Amy Cash Ahmed), my parents and, most especially, my wife Claudia. These people helped me by providing emotional support when I most needed it, and without them I surely would have faltered.

Although I tried to acknowledge as many of my collaborators as possible in each chapter, there are several individuals who were particularly instrumental in generating the data presented herein, and deserve special mention: Ian Traniello, Allyson Ray, Vikyath Rao and Tim Gernat. Over the course of many experiments (successful or otherwise) we became not just collaborators, but friends; thank you for your indispensable contributions and friendship.

I would also like to thank the faculty members who have served as my committee and mentors. Lisa, Rhanor Gillette, Justin, Craig and Gene gave me excellent guidance and support throughout my time as a student, and it goes without saying that this document would not be possible without them.

Last, but by no means least, a heartfelt thank you to the NSP staff. Stephanie Pregent was always there for support, whether it was friendly advice or to sign one of (many) forms after a missed deadline. As for Sam Beshers, what can I say? He has been a friend, confidante, running buddy, and even the officiant at my wedding. Thank you for cheering me through to the finish line yet again, Sam.

Table of Contents

Chapter 1: Endocrine Influences on Insect Societies ...... 1

Chapter 2: Insights into the Transcriptional Architecture of Behavioral Plasticity in the Honey Bee Apis mellifera ...... 104

Chapter 3: Behavioral state and social dynamics influence transcriptional regulatory network plasticity in the honey bee brain ...... 148

Chapter 4: Investigating the link between social networks and division of labor ...... 180

Chapter 5: Afterword ...... 206

Appendix A: Supplementary Figures ...... 208

Appendix B: Supplementary Tables ...... 232

Appendix C: Supplementary Datasets ...... 243

Chapter 1: Endocrine Influences on Insect Societies* Abstract Insect societies are defined by an intricate division of labor among individuals. There is a reproductive division of labor between queens and workers, and a division of labor among workers for all activities related to colony growth and development. The different castes in an insect society and the diverse roles they play are extreme manifestations of phenotypic plasticity. This chapter reviews the roles that various hormones play in governing different forms of division of labor in the insect societies, including juvenile hormone (JH), the ecdysteroids, insulin, biogenic amines, and neuropeptides. We discuss how these endocrine systems regulate diverse physiological and molecular processes during development and adulthood by serving as key signal transducers to combine information about internal and external state. We also draw on the results of a burgeoning literature on transcriptomic studies to propose a theoretical framework for how hormones modulate brain transcriptomic architecture underlying social behavior to generate phenotypic plasticity. A key feature of this framework is the notion that there has been neofunctionalization of certain endocrine systems via the rewiring of ancestral transcriptional regulatory networks. We end this chapter by presenting a mechanistic model for the evolution of insect sociality based on the co-option of endocrine pathways to respond to and regulate social behavior, using JH as a model system. In particular, we explore the relationship between the degree of neofunctionalization in JH-related pathways, the life stage at which JH modulates social stimuli, and the degree of phenotypic plasticity exhibited by various species.

1.0.0 Introduction Sociality is one of the most successful animal lifestyles. Many of the most dominant animal species on the Earth, as deﬁned by biomass and impact on their environment, are social (Hölldobler and Wilson, 1990; Wilson and Southwood, 1990). In the most extreme manifestations of sociality, a colony or society emerges from the collective properties of a group of individuals and their interactions. Insect societies in particular have long been an object of study and inspiration for biologists, because their collective behaviors provide an excellent example of how relatively simple organisms can interact to produce complex patterns of behavior. As a result, much research during the past century has been devoted to analyzing the collective behavior of social insects and the organization of their colonies (Wilson, 1971; Camazine et al., 2001; Hoelldobler and Wilson, 2008; Gadau and Fewell, 2009).

The apparently altruistic nature of these societies presents an additional enigma, as individuals sacrifice their own reproductive fitness to maximize the reproduction of one or more nestmates. At first glance this appears to be a contradiction of Darwinian theory, and efforts to solve this puzzle have had strong impact on the scientific community, influencing issues in the fields of developmental biology (Buss, 1987), sociobiology (Wilson, 1975), and evolutionary psychology (Wilson, 1998), as well as leading to the development of widely accepted theories of social evolution, including kin selection and the action of natural selection at multiple levels of biological organization (Crozier and Pamilo, 1996; Keller, 1999; Wilson and Holldobler, 2005; Hoelldobler and Wilson, 2008; Nowak et al., 2010).

Insects and vertebrates differ in many aspects of physiology, anatomy, and morphology, yet despite this disparity our understanding of social evolution has long suggested (Wilson, 1975) that the principles governing social behavior in these diverse taxa are strikingly similar. Since the last common ancestors of social insects and social vertebrates were solitary, it is therefore reasonable to expect that the evolution of sociality in both lineages involved at least some convergent selection on social behavior. If so, social evolution probably acted on conserved pathways, such as those that control how individuals respond to their environment and mechanisms related to nutrition (Toth and Robinson, 2007; Rittschof and Robinson, 2014, 2016) or the modulation of neural and behavioral plasticity by key (and sometimes conserved) neurochemical and

2 hormonal pathways (Ewards and Kravitz, 1997). Rittschof et al. (2014) have provided the ﬁrst support for this idea at the transcriptomic level by showing that the transcriptomic response to aggression-related contexts exhibits striking similarities in insects and vertebrates. Studying the evolution of mechanisms of social behavior beneﬁts from a comparative analysis within the social insects by taking advantage of the striking diversity of social systems they display across multiple independent origins of eusociality (Smith et al., 2008).

Many of the primary influences of hormonal signaling on cell physiology are mediated by the induction of transcriptional cascades either by the hormone’s receptor(s) or by intracellular signaling initiated by its receptor(s). The influence of these cascades on behavior can be divided into two general categories: organizational effects that guide the formation of the nervous system and activational effects that modulate the likelihood of specific behavioral outputs based on the organism’s internal state (Elekonich and Robinson, 2000). Elucidating these transcriptional cascades, and understanding how they interact, is critical to form a comprehensive picture of the dynamics of neuroendocrine function throughout the life span. The fact that the behavior of social insects involves the development of distinct morphological characteristics and behavioral programs makes them a uniquely suitable system for dissociating between the impact of endocrine cascades on developmental and proximal correlates of behavior.

As the ﬁrst social insect with a sequenced genome (Weinstock et al., 2006), the honeybee has provided the foundation for molecular inquiries into reproductive and worker division of labor. However, the recent publication of numerous additional social insect genomes has greatly expanded the utility of comparative molecular analyses at the genomic and transcriptomic levels. This chapter provides a review of the endocrinological, physiological, and molecular determinants underlying social behavior within the Hymenoptera, the insect order that contains the greatest number of species with colonial lifestyles. For a review of endocrine inﬂuences on the development and social behavior of termites, the other major group of social insects, see Watanabe et al. (2014).

1.0.1 Overview of Division of Labor in Insect Societies Insects display a range of social structures, from simple gregariousness to eusociality,

3 the most derived form of social organization. Eusociality is deﬁned by three traits: (1) cooperative care of the young by members of the same colony; (2) reproductive division of labor, with sterile individuals working on behalf of fecund relatives; and (3) an overlap of at least two generations of adults in the same colony (Michener, 1969; Wilson, 1971). The discovery of eusociality among several insect orders (Crozier and Pamilo, 1996; Choe and Crespi, 1997), as well as crustaceans (Duffy, 1996) and mammals (Jarvis, 1981), has led to the idea that eusociality represents an extreme version of the interplay between cooperation and competition that marks life in all animal societies, both invertebrate and vertebrate.

1.0.2 Division of Labor for Reproduction Division of labor for reproduction (whereby some society members reproduce much more than others) lies at the heart of eusociality. Hymenoptera display the haplodiploid mode of sex determination; fertilized, diploid eggs develop into females and unfertilized, haploid eggs develop into males. Complementary sex determination (csd) encodes a protein that serves as a master regulator of sexual development in the honeybee (Beye et al., 2003), and similar genes (all independently derived paralogs of the feminizer gene) are thought to govern sexual development in other hymenopteran species (Koch et al., 2014). The presence of two different csd alleles induces fertilized embryos to develop female sexual characteristics, whereas a single csd locus in haploid eggs (or homozygosity in diploid eggs) results in the development of a male. Although hormones are a primary factor in determining sexual development in vertebrates, it is not yet known whether csd or other genes involved in sexual development interact directly with endocrine factors. Since male hymenopteran social insects are involved only peripherally in the growth and maintenance of their colony (their only known role is to mate with virgin queens), this chapter deals primarily with the female colony members.

Female social Hymenoptera can develop into either queens (individuals with high reproductive potential) or workers (individuals with low reproductive potential). Depending on the species, these castes can arise as a consequence of endocrine- mediated alternative developmental processes during the larval period or emerge from ﬂuid dominance hierarchies established among individuals during adulthood. A queen has enlarged ovaries and a sperm-storing organ that can maintain viable sperm for the

4 duration of the individual’s life, while workers (which may number from tens up to millions in a colony) tend to be either completely or partially sterile, and perform most, if not all, tasks related to colony maintenance and growth (Wilson, 1971).

A queen may lay up to several thousand worker eggs per day, but ﬁtness in an insect society is determined by the production of new colonies via reproductive females and males. In some species, new queens leave their natal nest and attempt to found colonies solitarily, while in others, the entire colony splits into two and a fragment of the colony leaves to engage in group colony founding (Hölldobler and Wilson, 1990). Colony-level reproduction thus depends both on the reproductive activities of the queen and the activities of the workers that contribute to colony maintenance and growth, and only prosperous colonies can dedicate the resources to this process.

1.0.3 Division of Labor Among Workers In most insect societies the activities of the workers are also organized by a division of labor, with age-related division of labor being one of the most common forms (Robinson, 2002). In this system, workers follow a relatively predictable developmental trajectory that governs the order in which they specialize in particular tasks. They typically work inside the nest when they are young and shift to defending the nest and foraging outside as they age. In the more elaborate forms of age-related division of labor, such as in honeybee colonies, workers perform a sequence of jobs in the nest before they mature into foragers. Although this trajectory is relatively stable, diverse factors ranging from genotype to nutritional physiology to life history and social demography can modulate the speed of the transition or even reverse transitions that have already been made (Robinson, 1992; Ament et al., 2012a).

It is now well established that the stable but plastic behavioral states exhibited by social insects are associated with similarly stable changes in brain transcriptomic state (Zayed and Robinson, 2012). These changes are strong and reliable enough to allow for the accurate prediction of behavior based on the expression of only a subset of genes (Whitﬁeld et al., 2003; Toth et al., 2007; Alaux et al., 2009b; Liang et al., 2014), indicating that a direct and probably causal relationship exists between the brain transcriptome and worker division of labor. Moreover, cross-fostering studies between subspecies of bees with different behavioral proﬁles have revealed both inherited and environmental

5 inﬂuences on brain gene expression (Whitﬁeld et al., 2003; Alaux et al., 2009b). Therefore, a reciprocal relationship probably exists between the brain transcriptome and worker behavior, whereby genetically determined biases in gene expression are reinforced or attenuated based on the individual’s life history, leading to shifts in (or the stabilization of) behavioral state. Given that similar relationships have been found in vertebrates (Oliveira, 2009; Cardoso et al., 2015), it appears that social behavior more generally is governed by a four-part axis consisting of reciprocal interactions between neural circuitry, neurotranscriptomic state, endocrine signaling, and life history.

A more extreme form of division of labor among workers is based on differences in worker morphology that result from developmental processes during pre-adulthood, similar to worker–queen determination (Hölldobler and Wilson, 1990). There are clear examples of such morphologically distinct worker castes in many species of ants. Small ant workers typically labor in the nest while bigger individuals defend and forage. In many cases, this form of division of labor also involves elaborate morphological adaptations beyond simple differences in relative size, such as allometric growth of powerful jaws in castes specialized for defense. In all of the social hymenoptera, male and female pre-adults are relatively helpless and must rely on their older sisters for sustenance and protection.

1.0.4 Primitive and Advanced Eusociality Division of labor systems in the social Hymenoptera show great variation, but can broadly be grouped into one of three categories based on the level of flexibility exhibited within their division of labor: facultative eusociality, primitive (or ‘simple’) eusociality, and advanced (‘complex’) eusociality (Wilson, 1971; Michener, 1974; Fletcher and Ross, 1985; Bourke, 2011; Hunt, 2012). Facultatively, eusocial insects can create nests either solitarily or in small groups of two to a few dozen adults depending on species-specific ecological and possibly genetic factors. These colonies tend to be seasonal, and division of labor is maintained by direct behavioral interactions between the queen and the workers. Moreover, unlike most primitive and advanced eusocial species, the workers are capable of exhibiting levels of fertility on par with the queen if removed from her influence and can even mate to produce fertilized eggs. Since the molecular and neuroendocrinological basis of facultative eusociality is not well studied, these species will mostly be discussed in an evolutionary or speculative context. Fortunately, the

6 availability of newly sequenced genomes (Kocher et al., 2013; Kapheim et al., 2015) and transcriptomic studies (Jones et al., 2015) has laid the groundwork for future mechanistic analyses of behavior in these species.

Primitively eusocial insects share a number of key aspects with facultatively eusocial species, although these characteristics are generally more exaggerated or pronounced in the former (Wilson, 1971; Michener, 1974; Fletcher and Ross, 1985; Bourke, 2011; Hunt, 2012). Primitively eusocial insects establish small, temporary colonies with a few dozen to several hundred nestmates. Although there are few morphological differences between queens and workers, there are often substantial differences in physiology, size, and fertility. As in facultatively eusocial insects, division of labor for reproduction is achieved by a dominance hierarchy (with the queen as the alpha individual) that is established and maintained with direct behavioral mechanisms, including pushing, biting, and the physical prevention of egg laying. The inhibitory inﬂuence of the queen on worker reproduction is best illustrated by increased oogenesis and oviposition behavior by workers following her removal or death. Behavioral domination is an ongoing process, because adult females are generally capable of producing offspring (at least male progeny), albeit at a lower rate than the queen. As a consequence, life in primitively eusocial colonies is by necessity characterized by a relatively high degree of aggression to maintain the queen’s reproductive rights.

Division of labor among workers in primitively eusocial species also appears to be less structured than in advanced eusocial species, although there is evidence for weak age- related division of labor in some species (Naug and Gadagkar, 1998; Cuvillier-Hot et al., 2001; Giray et al., 2005; Lengyel et al., 2007). In many species there is a strong relationship between position in a reproductive dominance hierarchy and labor proﬁle. Higher-ranked individuals tend to work inside the nest, where it is safer and less physically demanding, while lower-ranked individuals perform the riskier foraging tasks (e.g., West-Eberhard, 1975; Gadagkar, 1991; Jeanne, 1991a; and Powell and Tschinkel, 1999).

In advanced eusocial species, colonies are typically perennial and populations number in the thousands to millions of individuals. Queens and workers are distinguished by striking morphological differences due to preadult processes of caste determination. This

7 is itself a fundamental form of reproductive division of labor, as it results in castes that are physically incapable of fully performing one another’s tasks. For instance, honeybee workers have highly deﬁcient ovary development and lack the ability to mate and store sperm, while queens have a proboscis that is poorly suited to nectar gathering and lack corbicula (the structures used by foragers to transport pollen).

In advanced eusocial species, queen inhibition of worker reproduction is achieved largely by chemical communication rather than by direct physical aggression (Slessor et al., 1998; Le Conte and Hefetz, 2008; Holman et al., 2010; Oliveira et al., 2015). Since the discovery and characterization of the ﬁrst queen pheromone in the honeybee (Slessor et al., 1998), inhibitory primer pheromones have been identiﬁed in other advanced eusocial insects, including ants (Holman et al., 2010), wasps (Van Oystaeyen et al., 2014), and other corbiculate bees (Oliveira et al., 2015). Although these systems appear to mainly use cuticular hydrocarbons rather than pheromones produced by mandibular glands (as in the bee), the similarities in pheromone composition across species have led to the hypothesis that queen pheromones may be derived from deeply conserved fertility signals that predate the evolution of eusociality (Oi et al., 2015), an idea that has found some support in phylogenetic analyses (Oliveira et al., 2015).

The adaptive signiﬁcance of these pheromones can be argued based on the rationale that in larger societies, it is inconceivable that the queen can regularly interact with all workers in the colony. These diffuse signals therefore allow her to communicate with or control individuals that she would not otherwise have physical contact with. Although queen pheromones traditionally have been thought to exert direct inhibitory effects on worker reproductive physiology and behavior, Keller and Nonacs (1993) suggested instead that queen pheromones signal the presence of a reproductively dominant queen to workers, which then leads to autoinhibition. Decisive evidence in favor of either perspective is lacking, in part, because experiments distinguishing between these two hypotheses are difﬁcult to design (Slessor et al., 1998; Le Conte and Hefetz, 2008).

Worker–worker interactions also play a vital role in maintaining reproductive division of labor in advanced eusocial species. During larval and pupal development, differential provisioning of the brood by nurses results in caste determination in many advanced eusocial species. During adulthood, workers will also ‘police’ one another by selectively

8 removing worker-laid eggs (Ratnieks and Visscher, 1989) and exhibiting aggression toward workers with activated ovaries (Visscher and Dukas, 1995). Moreover, the presence of brood itself can inﬂuence the reproductive development of both workers and queens (e.g., Kropacova and Haslbachova, 1971; Jay and Jay, 1976; and Tschinkel, 1995). The endocrine basis of worker-mediated inhibition of reproduction is not well understood, so our discussion will not address this particular aspect of reproductive division of labor.

1.1.0 Insect Hormones that Influence Division of Labor A number of hormones have been implicated in the control of division of labor in insect societies: juvenile hormone (JH), the ecdysteroids, insulin-like peptides (Ilp), and neuromodulators such as the biogenic amines and various neuropeptides. Vitellogenin (Vg), best known as a yolk protein, also appears to act as a peptide hormone in the regulation of division of labor in at least some species (Page et al., 2012). Together, these molecules control diverse physiological and behavioral phenomena in insects, including metamorphosis, reproduction, pheromone production, diapause termination, and behavior (Nijhout, 1994; Gilbert, 2012).

The JHs are a family of sesquiterpenes (with JH-III being the active form in Hymenoptera) that are structurally most similar to retinoic acid, a morphogenic molecule in vertebrates. As is the case for hormones in the pituitary/hypothalamus axis in vertebrates, production of JH by the corpora allata is regulated both by circulating factors and by direct innervation via neurosecretory cells located in the pars interecerebralis (PI) (Tobe and Stay, 1985; Goodman et al., 2005). In adult insects, JH generally functions as a gonadotropin (Nijhout, 1994; Wyatt and Davey, 1996; Gilbert, 2012), either directly causing or hastening the onset of reproductive maturation. In this respect, JH regulates Vg synthesis by the fat body (Seehuus et al., 2007) and its uptake by developing oocytes, as well as the synthesis of sex pheromones and accessory gland products (Tillman et al., 1999). Vg has also been recently identiﬁed in the brain of adult honeybees (Munch et al., 2015), but the role that JH plays in this phenomenon is unknown.

JH has been implicated in the control of reproductive behaviors, such as female receptivity (Ringo et al., 2002), pheromone release (Cusson and McNeil, 1989), and oviposition (Strambi et al., 1997), as well as maternal behaviors (Trumbo et al., 2002),

ﬂight activity (de Oliveira Tozetto et al., 1997), and feeding (Schal et al., 1997). JH may inﬂuence behavior, at least in part, by its effects on response thresholds, as its presence can increase the excitability of sensory interneurons in at least some species (Anton and Gadenne, 1999; Stout et al., 2002). Much of this chapter revolves around the roles of JH in reproductive and worker division of labor, as well as in the evolution of eusociality in a general sense. In part, this is because JH has been shown to play a pivotal role in so many processes central to division of labor. However, it is also by far the best-studied hormone in the social insects, leading to the potential for overemphasizing its importance. Our perspective is that JH is not necessarily the primary endocrine factor governing division of labor, but operates as part of a constellation of hormones that interact to produce the various forms of phenotypic plasticity that characterize the insect societies.

Ecdysteroids are a family of steroid hormones; the most common form in insects is 20- hydroxyecdysone (henceforth referred to as ‘ecdysone’), which is converted from ecdysone in the fat body or epidermis (Nijhout, 1994). Ecdysteroids are produced by the prothoracic glands in immature insects and by the gonads in adults. Like other steroids, ecdysteroids can pass through the cell membrane, enter the nucleus, and bind to nuclear receptor proteins; the receptor–hormone complex then binds to speciﬁc sequences of the DNA to regulate gene transcription. In adult insects, ecdysteroids are involved in the regulation of oogenesis, vitellogenesis, and sex pheromone production (Hagedorn, 1985) and have been implicated in the control of reproductive behavior (Nijhout, 1994; Ringo et al., 2002). Ecdysteroids, such as vertebrate steroid hormones (Wehling, 1994), may have rapid, nongenomic actions on cell membrane properties (Ruffner et al., 1999), which might be particularly relevant for understanding the role of ecdysteroids as behavior modulators.

As in vertebrates, nutritional and metabolic states are communicated in large part by the release of peptide hormones. Ilp, the invertebrate ortholog of insulin, is produced in cells throughout the insect body, particularly in the fat bodies (the insect analog of the liver) and the PI region of the brain. Insulin and insulin-related signaling has been linked to division of labor in numerous contexts and species, indicating that nutritional state plays a central role in regulating division of labor.

The biogenic amines, particularly dopamine, serotonin, and octopamine (the insect

10 analog of norepinephrine), inﬂuence diverse physiological and behavioral processes in both vertebrates and invertebrates by acting as neurotransmitters, neuromodulators, and neurohormones (e.g., Kravitz, 1988, Menzel and Muller, 1996; and Roeder, 1999). Although biogenic amines are generally produced in the brain, other organs (including the ovaries) are thought to produce biogenic amines as well. In social insects, biogenic amines often interact with other endocrine systems, modulating their inﬂuence on nervous system function and behavior in different castes and subcastes.

1.1.1 Endocrine Signaling in Reproductive Division of Labor Endocrine pathways appear to play a vital role in nearly every aspect of division of labor in the social insects. This section details the importance of endocrine signaling and endocrinemediated gene expression in establishing and maintaining caste-speciﬁc differences in reproduction. As will be discussed below, these effects can be established either during preadult development or as a function of life history and the colony environment. The relative importance of such organizational or activational effects depends largely on the degree of ﬂexibility in reproductive division of labor exhibited by the society (that is, whether they exhibit primitive or advanced eusociality).

Molecular and endocrine studies of caste determination have relied primarily on whole larvae analyses, rather than analyses of nervous tissue. However, the presence of striking differences in genes related to neurotransmitter signaling and nervous system development between castes (Barchuk et al., 2007) implies that systemic changes in transcription and physiology (most likely induced in large part by endocrine signaling) play critical roles in organizing nervous system function during caste development. This parallels what has been found in other holometabolous insects, where diverse endocrine signals are involved in remodeling the nervous system during metamorphosis (Williams and Truman, 2005; Boulanger and Dura, 2015). These results point to as yet unidentiﬁed interactions between hormones and the developing nervous system to govern the dramatic differences in caste-speciﬁc behaviors observed in adult queens and workers. In addition to the endocrine and social factors described below, abiotic factors (such as temperature or seasonality) can play a vital role in caste determination in ants (Hölldobler and Wilson, 1990) and halictid bees (Choe and Crespi, 1997). However, the connection between these factors and endocrine signaling in the context of division of labor is not well studied, so we will not discuss them further.

1.1.2 JH, Vg, and Reproductive Division of Labor The link between JH and reproductive behavior in primitively eusocial insects has been best studied in the bumblebee, Bombus terrestris (unless otherwise speciﬁed, all further references to bumblebees in this section relate to this species). Bombus terrestris is a temperate species with an annual colony life cycle that exhibits three phases of organization: (1) the founding phase, when the queen initiates nest construction and brood rearing; (2) the ergonomic phase, a time of rapid increase in colony size via the production of sterile workers; and (3) the competition phase, when reproductive females (gynes) and males are produced. This last phase is characterized by aggression between the workers and the queen and the subsequent establishment of dominance hierarchies based largely on size (Duchateau and Velthuis, 1988).

Fecundity is controlled, at least in part, by JH, which functions as a gonadotropin (Figure 1.1). A strong, positive correlation exists between oocyte length and JH titers, independent of age and social conditions (Bloch et al., 2000a). Egg-laying individuals have the highest hormone titers, and JH treatment of workers causes a dose- dependent increase in oocyte length even in the presence of the queen (Röseler, 1977; van Doorn, 1987, 1989; Röseler and Röseler, 1988). Further, surgical removal of the corpora allata (allatectomy) (Shpigler et al., 2014) or the pharmacological inhibition of JH production (Amsalem et al., 2014b) induces drastic decreases in wax production, Vg synthesis, ovary development, and egg-laying behavior in queenless workers. These deﬁcits can be rescued by JH-III administration, indicating that JH coordinates multiple processes associated with female reproduction in bumblebees (Shpigler et al., 2014).

Bumblebees also provide a clear demonstration of the sensitivity of JH titer to social context. Changes in JH biosynthesis rates occur rapidly in response to the presence or absence of social stimuli and can be detected within 1 day of queen removal (Röseler and Röseler, 1978), inducing concomitant changes in JH titer as soon as 3 days postremoval (Bloch et al., 2000a; Figure 1.1). This endocrine sensitivity has been exploited to uncover the previously overlooked role of inhibitory worker– worker interactions (Bloch and Hefetz, 1999a; Bloch et al., 2000a), whereby older workers inhibit the reproductive development of younger workers, particularly after the onset of the colony’s reproductive

12 phase. Although JH appears to generally serve as a gonadotropin in primitively eusocial paper wasps such as Polistes as well, exceptions do exist (Table 1.1).

In addition to serving directly as a gonadotropin, JH may also inﬂuence reproductive division of labor by increasing the frequency of aggressive behaviors that establish and maintain an individual’s position within the dominance hierarchy. This idea is consistent with the general association between high JH levels, developed ovaries, aggression, and high social rank in paper wasp and bumblebee societies. This link between reproductive rights, social dominance, and endocrine signaling corresponds with basic themes in vertebrate behavioral endocrinology (e.g., Monaghan and Glickman, 1992), and it has been suggested that JH may mediate aggression in response to social challenge in a manner similar to testosterone in vertebrates (Tibbetts and Huang, 2010).

However, the extent and generality of JH’s influence on social insect aggression is not clear. For example, although dominant queenless bumblebee workers have higher rates of JH biosynthesis than do subordinates (Larrere and Couillaud, 1993; Bloch et al., 2000a), it is unclear whether JH directly influences dominance given that JH peaks after a dominance hierarchy has already been established (Röseler and Röseler, 1978; Bloch et al., 1996, 2000a). Moreover, JH treatment does not increase worker dominance in either queenless or queenright conditions (van Doorn, 1987, 1989). However, the recent finding that administration of precocene I, which inhibits JH synthesis, can reduce aggression in queenless bumblebees (Amsalem et al., 2014b) implies that there may be some link between JH and dominance behavior, even if exogenous treatments of JH cannot enhance aggression.

Many species in the Polistes genus also exhibit a strong, positive relationship between JH biosynthesis and position within the dominance hierarchy (Roseler, 1991). However, although JH administration increases dominance behaviors in some species (Table 1.1), it appears that the precise behavioral effects are heavily dependent on physiological and nutritional state (Tibbetts and Izzo, 2009; Tibbetts and Huang, 2010; Tibbetts et al., 2011), as well as social interactions with nestmates (Tibbetts and Sheehan, 2012). In fact, some Polistes species do not exhibit a link between JH and dominance at all (Table 1.1).

The endocrine basis of reproductive division of labor in advanced eusocial societies has been best studied in the European honeybee, Apis mellifera. Honeybees live in perennial colonies containing tens of thousands of individuals headed by a single queen (Winston, 1987), and colony reproduction occurs via ﬁssioning. Like other advanced social insects, reproduction is strongly biased toward the queen, with the workers developing their ovaries only rarely. Under normal conditions, there is no aggression between the queen and the workers Factors known to inhibit worker reproduction include a complex blend of queen pheromones produced by different exocrine organs (mandibular glands, tergal glands, and Dufour’s gland) as well as by pheromones emitted from the brood (Slessor et al., 1998; Le Conte and Hefetz, 2008).

JH does not appear to be a gonadotropin in honeybees, as it has no known role in oogenesis or in the onset or maintenance of vitellogenesis in adult bees. JH titers are generally very low in adult queens, much lower than those in worker bees that act as foragers (see Figure 1.2). Allatectomized queens show almost normal patterns of Vg incorporation and egg-laying behavior (Engels and Ramamurty, 1976; Ramamurty and Engels, 1977; Daerr, 1978). In addition, JH treatment fails to stimulate Vg production in vitro (Engels et al., 1990). However, JH does affect vitellogenesis and ovarian development during the 5th larval instar and pupal stages (Capella and Hartfelder, 1998; Barchuk et al., 2002), as well as during the ﬁrst few days of adulthood prior to vitellogenesis and mating (Fahrbach et al., 1995), suggesting that JH inﬂuences the ontogeny of the reproductive system, but not its maintenance.

This disconnect between JH and fertility extends to worker honeybees. Worker honeybees from typical colonies with a queen and brood show very little, if any, reproductive development (e.g., Visscher, 1989), despite JH titers that increase drastically during the onset of foraging behavior (Robinson and Vargo, 1997). Further, increases in JH titers are not positively related to oogenesis during adulthood (Hartfelder and Engels, 1998). Studies conducted with colonies where both queen and brood have been removed, resulting in the development of workers with active ovaries, similarly revealed that egg-laying workers have low JH titers (Engels et al., 1990; Robinson et al., 1991; Hartfelder and Engels, 1998).

Not only does JH fail to act as a gonadotropin in honeybees, but the canonical

14 relationship between Vg and JH, in which JH positively regulates Vg production and accumulation into developing oocytes, is altered in adults. Although Imboden and Luescher (1975) and Imboden et al. (1976) ﬁrst reported that Vg synthesis was abolished in allatectomized honeybee workers, later studies indicate that the relationship between Vg and JH is not so simple: low doses stimulate Vg synthesis, but high doses substantially inhibit it (Imboden et al., 1976; Rutz et al., 1976; Pinto et al., 2000). Similarly, JH analog treatment causes a downregulation of vg RNA, in both workers and queens (Corona et al., 2007). vgRNAi treatment leads to a corresponding increase in JH, suggesting that Vg, in turn, downregulates JH (Amdam and Omholt, 2003; Amdam et al., 2007). Since JH and Vg titers do follow the expected trend during larval development (Barchuk et al., 2002; Guidugli et al., 2005), this inhibitory feedback loop between JH and Vg (Amdam and Omholt, 2003; Amdam et al., 2007) may represent an evolutionary novelty that allows for the decoupling of each molecule’s classic role in the regulation of reproduction, allowing them to modulate worker division of labor instead (see Section 1.2.1).

Unlike JH, it is thought that the traditional relationship of Vg to oogenesis exists in honeybees (although Vg also has functions independent of oogenesis and reproduction in worker bees; see Section 1.2.1). Comparisons between bee strains selected for pollen or nectar foraging preferences suggest a correlation between high Vg and reproductive potential. Workers from strains selected for high pollen foraging have higher Vg titers, higher JH titers at 1 day of age, and increased reproductive potential (more ovarioles) and are more likely to lay eggs under queenless conditions, relative to low pollen foraging strains (Amdam et al., 2006a, 2007). However, strains of honeybees selected for high levels of worker reproduction and ovary development failed to show the expected preference for pollen (Oldroyd and Beekman, 2008). These ﬁndings suggest that the links between ovariole number and foraging behavior are not obligate and that more research is needed to determine how general this relationship is in honeybees.

As expected based on previously ascertained differences in hormonal titers between queen and worker preadults (Hartfelder and Engels, 1998), the expression of enzymes related to JH synthesis is upregulated in queens and peaks around the 3rd larval instar, shortly before caste is determined. Caste-speciﬁc differences in the expression of the JH receptor Methoprene Tolerant (met) and the JH-responsive transcription factor

Kruppel Homolog 1 (kr-h1) are not detectable until after the 4th larval instar, at which point queen-destined larvae show a large increase in expression that persists until the beginning of pupation (Hartfelder et al., 2015). This points to the existence of a threshold effect for JH’s inﬂuence on downstream gene expression. Presumably, once a critical level of JH is reached, it results in the propagation of transcriptomic cascades, through endocrine-responsive transcription factors such as met and kr-h1, that results in the development of queen morphology and the prevention of ovary apoptosis. This model is supported by the fact that manipulation of JH-responsive genes downstream of these signaling factors (such as the storage protein hexamerin 70b) can induce queenlike morphology in worker-destined bees (Cameron et al., 2013). In almost all primitively eusocial bumblebees and paper wasps studied thus far, JH acts as a gonadotropin, while, in the advanced eusocial honeybee, it apparently does not. Might this be the beginning of a heuristic dichotomy?

Studies of fire ants suggest that the situation is not so simple. Fire ants are also an advanced eusocial species, with large, perennial colonies, one or more fully reproductive queens, and intricate systems of division of labor (Fletcher and Ross, 1985; Hölldobler and Wilson, 1990). Division of labor for reproduction in this species is particularly rigid, as workers lack functional ovaries and are completely sterile (Hölldobler and Wilson, 1990). Colony reproduction involves the production of hundreds of winged virgin queens that leave the colony en masse, mate, shed their wings, and attempt to find colonies solitarily. Fletcher and Blum (1981) fortuitously discovered that virgin queens are reproductively inhibited by the mated queen (or queens) in their colony, causing them to retain their wings and show no ovary development in the presence of a queen. The social inhibition of wing shedding and reproduction allowed for a detailed dissection of the endocrine correlates of reproductive behavior and the finding that JH plays a key role in the regulation of wing shedding and oogenesis in fire ants.

JH biosynthesis rates and whole body content are signiﬁcantly higher in egg-laying queens compared with workers or virgin queens. Following isolation, virgin queens show a rapid increase in both JH content and in vitro biosynthetic rates, with a ﬁrst peak at about 3 days in isolation that coincides with the time of wing shedding (Brent and Vargo, 2003). Treatment with JH or a JH analog induces both wing shedding and egg laying (Kearney et al., 1977; Vargo and Laurel, 1994; Burns et al., 2002). Removal of the

16 corpora allata (Barker, 1978, 1979) or treatment with precocene (Burns et al., 2002) blocks these effects even in individuals isolated from their queen, and hormone replacement reversed them. The JH analog methoprene also upregulates a Vg receptor that is expressed specifically in the ovaries of reproductive female virgin alates and queens (Chen et al., 2004). Although these findings are consistent with the hypothesis that JH modulates reproduction in fire ants, the relationship between JH and ovary activation in isolated virgin queens is complex, with several peaks in JH despite an almost continuous temporal increase in the number of vitellogenic oocytes (Brent and Vargo, 2003).

As expected in an advanced eusocial species, the proximate mechanism that normally prevents virgin queen ﬁre ants from initiating reproductive maturation is a primer pheromone produced by the reigning queen (Fletcher and Blum, 1981; Vargo and Laurel, 1994; Vargo and Hulsey, 2000). The putative ﬁre ant queen pheromone acts by suppressing JH titers, thus preventing ovary development by blocking Vg uptake by the ovaries. Vg synthesis does not seem to be directly affected, however, as high levels of Vg are present even though the ovaries remain undeveloped. One hypothesis to explain these effects is that the putative pheromone regulates JH production so that JH titers are below the threshold needed to stimulate Vg uptake by the ovaries, but still above the level required to induce Vg synthesis by the fat body (Robinson and Vargo, 1997). Alternatively, different hormones may control Vg synthesis and vitellogenesis (Nijhout, 1994). The mechanism linking JH with wing shedding is also unknown, but Vargo and Laurel (1994) proposed that JH may act directly on the nervous system to elicit this behavior.

Similar pheromone and endocrine processes may explain how mated queens inhibit each other in colonies with more than one queen (Vargo, 1992), a well-documented phenomenon in the social insect literature (Vargo and Fletcher, 1989; Jeanne, 1991b; Passera et al., 1991; Ross and Keller, 1995). However, the endocrine basis of interactions among mated queens has not received much attention (Martinez and Wheeler, 1991).

In summary, there is substantial information to implicate JH in the regulation of division of labor for reproduction, but the involvement of this hormone appears to be limited to

17 only some, not all, species of social insects studied to date (Table 1.1). The fact that this variability occurs within individual clades strongly suggests that the loss of JH regulation of oogenesis has occurred several times during the evolution of hymenopteran societies.

1.1.3 Ecdysone and Reproductive Division of Labor JH interacts with ecdysteroids in a variety of contexts in insect biology, including metamorphosis (Nijhout, 1994), ovarian development (Hagedorn, 1985), and, potentially, behavior (Pandey and Bloch, 2015). It is therefore reasonable to expect the possible involvement of these hormones in the regulation of division of labor for reproduction in social insects. However, the function of ecdysteroids in the reproductive physiology of social insects is still not well understood, possibly because ecdysteroids regulate multiple physiological and developmental processes that are not always identical in all species (De Loof et al., 2001; Raikhel et al., 2005). In general, understanding the role of a hormone in social insect reproduction is difficult because it also is influenced by complex socially mediated processes such as caste development and dominance hierarchies. Unfortunately, only a few studies have been conducted on ecdysteroids in the context of social insect reproduction. The main source for ecdysteroids in workers and queens appears to be the ovaries (Hagedorn, 1983, 1985; Nijhout, 1994). Ovariectomy reduced the ecdysteroid titer to basal levels in paper wasp nest foundresses (Röseler et al., 1985), and activated ovaries possess the highest levels of ecdysteroids in honeybees (Feldlaufer et al., 1986), bumblebees (Geva et al., 2005), and paper wasps (Strambi et al., 1977). However, there is only a weak correlation between oocyte stage and hemolymph ecdysteroid titer in most of the species studied thus far (Strambi et al., 1977; Röseler et al., 1985; Kaatz, 1987; Robinson et al., 1991; Bloch et al., 2000b; Hartfelder et al., 2002; Geva et al., 2005). This may reflect the dynamics of ecdysteroid secretion from the ovary to hemolymph or variation in ecdysteroid degradation during the period of ovarian development.

The relationships between ecdysteroids, social organization, and reproductive physiology in social insects are complex. In paper wasps, ecdysteroids and JH are associated with reproductive dominance and oogenesis (Strambi et al., 1977; Röseler et al., 1984, 1985; Röseler, 1985). Ecdysteroid titers are also correlated with ovarian state and dominance rank in the queenless ant Streblognathus peetersi, a species in which JH does not appear to function as a gonadotropin. In this species, the most dominant worker

18 has the highest ecdysteroid titer but the lowest rates of JH biosynthesis in vitro. Moreover, ecdysteroid titer decreases (and JH biosynthesis rate increases) along a linear dominance hierarchy (Brent et al., 2006). In bumblebees, egg-laying queens heading colonies and dominant queenless workers have the highest levels of ecdysteroid in both the hemolymph and ovaries, though there is not necessarily a direct connection between ecdysone titer and the number of eggs laid (Bloch et al., 2000b; Geva et al., 2005). Therefore, the emerging picture for primitively eusocial bees and wasps, and for queenless ants, is that ecdysteroids are closely associated with the ovarian state, as in many nonsocial insects (Hagedorn, 1983, 1985; Nijhout, 1994).

The situation in advanced eusocial hymenopterans is less clear. Robinson et al. (1991) reported that ecdysteroid titers in honeybees were undetectably low in queenright workers performing nursing or foraging activities. Ecdysteroid levels were higher in queenless egg-laying workers and higher still in laying queens. These results suggest that ecdysteroids play a role in the regulation of reproduction in honeybees, as they do in bumblebees. By contrast, Hartfelder et al. (2002) found that queenright honeybee workers did have detectable levels that were typically as high as in queenless workers, but at an earlier age than sampled in Robinson et al. (1991). Hartfelder et al. (2002) also found no differences between virgin and laying queens. In the stingless bee Melipona quadrifasciata, however, ecdysteroid titers were undetectably low in both mated and unmated queens, higher in queenless workers, and highest in queenright workers (Hartfelder et al., 2002).

Treatment studies have thus far failed to clarify the situation. Ecdysteroid treatment stimulated Vg production in an in vitro preparation of fat bodies from queen honeybees (Engels et al., 1990), but ecdysteroid treatment in vivo delayed the onset of Vg synthesis during pupal development in both queens and workers (Barchuk et al., 2002). This ﬁnding is consistent with a decline in ecdysteroid titers during this stage in pupal development (Feldlaufer et al., 1985; Hartfelder and Engels, 1998). One possible interpretation for these apparently contradicting results is that the inﬂuence of ecdysteroids on vitellogenesis and reproductive physiology changes during bee development, as suggested earlier for JH. Hartfelder et al. (2002) further suggested that ecdysteroids, such as JH, have lost their gonadotropic role during the evolution of advanced eusocial bees, an interesting hypothesis that needs to be tested further (but see below for

19 evidence linking ecdysone-responsive genes to fertility).

The social environment appears to have diverse and complex influences on ecdysteroid levels, best exemplified in studies with bumblebees. The presence of a reproductively active queen appears to be the most significant social factor. Workers from colonies with an egg-laying queen had the least developed ovaries and the lowest ovarian ecdysteroid content, low hemolymph ecdysteroid titers, and rarely showed aggressive behavior. Ecdysteroid levels were also low in subordinate workers in queenless groups or colonies (Bloch et al., 2000b; Geva et al., 2005).

There is little empirical work on the physiological functions of ecdysteroids in social hymenopterans. It has been suggested that ecdysteroids are involved in the control of dominance behavior, but again the pattern is not consistent across all social insect species. In bumblebees, high ecdysteroid titers are correlated with dominance rank among workers in small queenless groups, as well as in orphan colonies (Bloch et al., 2000b; Geva et al., 2005). However, in spite of this association, high ecdysteroid titers are probably not a prerequisite for the expression of aggressive and dominant behavior, because ovariectomy has no effect on dominance, even though egg laying and eggcup construction are eliminated (van Doorn, 1987, 1989). One possibility is that levels of ecdysteroids are sensitive to social rank, but not vice versa. In paper wasps, ecdysteroid treatment, alone or with JH, increased the likelihood of displaying dominance behavior in assays involving pairs of foundress queens (Röseler et al., 1984). However, ovariectomized foundresses were still able to become dominant females (Röseler et al., 1985; Roseler, 1991), indicating that ecdysone is probably not required to induce dominance or aggressive behavior. Another interesting possibility is that ecdysteroids inﬂuence egg laying rather than fertility per se, which may account for the association of high ecdysteroid titers with developed ovaries in paper wasps, bumblebees, and queenless ants.

Whole-body transcriptomic analyses of sterile and laying honeybee workers collected from a queenless colony in a natural setting (Cardoen et al., 2011) revealed large differences in the expression of ecdysone and JH-responsive transcription factor genes, such as E74, E93, broad, and ftz-f1. These results suggest that JH and ecdysone may play some role in the transition from sterility to reproduction. Genes encoding enzymes

20 required for the synthesis of Vg and ecdysone were also found to be upregulated in laying workers, and strong signatures of ecdysone-related gene expression (including ftz-f1) were observed in strains of bees that have more developed ovaries (Wang et al., 2012b). Similarly, genes in the ecdysone/JH axis (including HR46) have also been implicated by genetic screens of variation in worker ovary development (Page and Amdam, 2007) and were later conﬁrmed to exhibit changes in expression that closely track ovary development (Wang et al., 2009; Ronai et al., 2015).

1.1.4 Nutrition and Metabolic Factors Influencing Reproductive Division of Labor Although it is possible to modulate the fertility and reproductive behavior of adult workers in at least some species of social insects by altering their diet (Matsuyama et al., 2015), the vast majority of the literature examining the role of nutrition in reproductive division of labor centers on caste determination during larval development. The endocrine systems of insect larvae integrate information about both nutritional status and external environment (Nijhout and Wheeler, 1982). Nutrition plays a central role in caste determination in many social Hymenoptera (Wheeler, 1986), where it is transduced into an endocrine signal to influence developmental plasticity. Worker honeybees produce two blends of brood food from glands located in their heads, one type (royal jelly) richer in glandular secretions than the other (Winston, 1987). All larvae receive royal jelly for the first 2 days of their lives, but queendestined larvae continue to receive royal jelly throughout larval development, while worker-destined larvae receive less nutritious food (Laidlaw and Page, 1997). It has been known for some time that this dietary difference plays an important role in caste determination in honeybees, but establishing which features of the diet induce larvae to follow one developmental pathway or the other has proven difficult (Hartfelder and Engels, 1998). However, the recent detection of pharmacologically active compounds in worker and royal jelly that can alter growth, epigenetic factors, and caste-related gene expression, such as royalactin (Kamakura, 2011), 10-HDA (Spannhoff et al., 2011), and p-coumaric acid (Mao et al., 2015), has begun to provide insight into the molecular basis for nutritional effects on caste determination.

Gene expression differences related to metabolism and metabolic signaling (Cristino et al., 2006) also appear to play an especially crucial role in queen development. This includes the cytochrome P450 (CYP) gene family, with numerous members differentially

21 expressed or alternatively spliced in queenor worker-destined larva, depending on the stage of development being assayed (Evans and Wheeler, 2001; Cameron et al., 2013). CYP genes are known to be involved in hormone synthesis (Bernhardt and Waterman, 2007) and energy metabolism, which is upregulated in queens throughout development and adulthood (Begna et al., 2011; Cameron et al., 2013). Although it is not yet known what regulates CYP genes during caste determination, the strong association between caste determination and endocrine-related genes that characteristically regulate mitochondrial function is very suggestive of a hormonal inﬂuence (see below).

Insulin-IGF signaling (IIS) and the related insulin receptor substrate (IRS) signaling and target of rapamycin (TOR) pathways have been strongly implicated in caste determination via transcriptomic studies (Barchuk et al., 2007; Chen et al., 2012). As noted previously, the nutritional content of the food given to workerand queen-destined larvae differs substantially in honeybees and plays an important role in caste determination. Therefore, it is not surprising that there are caste-specific differences in the expression of ilp-1 and ilp-2 (Wheeler et al., 2006) and insulin receptors 1 and 2 (InR- 1 and InR-2) (de Azevedo and Hartfelder, 2008) during caste determination. Molecular dissections of the IIS and INR pathways, mostly via RNAi, suggest complex modular effects on various caste-related traits. For example, ilp-1 and ilp-2 influence JH and ovarian development, respectively, but neither has a substantial impact on the development of caste-specific morphology (Wang et al., 2013a). Similarly, perturbing the expression of an InR gene does not alter adult size or ovary development (Kamakura, 2011). Tor (Patel et al., 2007) and Egfr (Kamakura, 2011) are, however, involved in the regulation of various workerlike traits. These results have led to a model whereby IRS, Tor, and Egfr serve as the primary nutrition and metabolic sensors influencing caste determination, while IIS plays a modulatory role (Hartfelder et al., 2015).

The importance of such modulatory interactions is exempliﬁed by the molecular changes that occur during larval growth prior to caste determination. Experimentally switching larvae between cells used to rear either workers or queens revealed that all larvae are totipotent until about 3 days after hatching (Weaver, 1957), when a JH peak in queen- destined larvae induces caste speciﬁcation immediately prior to the 4th larval instar (Figure 1.2; Wirtz, 1973; Rachinsky et al., 1990). However, drastic differences in gene expression between queenand worker-destined larvae can be detected in as few as 6 h

22 after hatching and persist throughout larval and pupal development (Cameron et al., 2013). This suggests that gene expression is biased toward queenor workerlike patterns long before a decision point is actually reached. It has been hypothesized that these biases are the direct consequence of nutritional signaling cascades and serve to promote a particular developmental pathway until canalization is achieved, i.e., when the larva commits to becoming a worker or a queen (Wheeler and Robinson, 2014). This hypothesis is consistent with ﬁndings that IRS and Tor RNAi knockdowns lower JH levels (Mutti et al., 2011) and experimental switches from a queen diet to a worker diet alter gene expression proﬁles of numerous IRS and TOR pathway constituents (Wheeler and Robinson, 2014).

Much less is known about the inﬂuences and interactions of nutritional and endocrine factors on caste determination in other species. Intriguingly, reproduction in the harvester ant Pogonomyrmex rugosus also appears to involve a complex interaction of insulin signaling, JH, and Vg signaling (Libbrecht et al., 2013). Together with the correlation between IIS and body size and reproduction in Solenopsis invicta (Lu and Pietrantonio, 2011), these ﬁndings suggest that co-option of nutrient signaling is a common theme in the evolution of reproductive division of labor in the social Hymenoptera. In bumblebees, however, differences in the quality of nutrition do not appear to be a factor in caste determination (Pereboom, 2000), and the larger size of queen bumblebees is mostly due to a longer larval feeding period as larvae had heavier weight at the time of molting (Cnaani et al., 1997). The basis for the lengthening of the feeding period involves direct interactions between the queen and developing larvae (Shpigler et al., 2013).

Although substandard nutrition can prevent queen fate speciﬁcation in the Meliponini (stingless bees), the genetic background of the larvae is thought to be the key factor in caste determination (Kerr, 1969). However, the recent discovery that geraniol in the labial gland secretions of nurses can increase the rate of queen development suggests that socially mediated (and potentially nutrition-related) factors may still play a role in caste determination in this group of bees as well (Jarau et al., 2010). The fact that fecundity and body size in queens also is inﬂuenced by genotype in honeybees (Linksvayer et al., 2011) suggests that caste determination in all social insects involves an interaction of heritable, social, and nutritional factors, but that the relative importance of each factor varies from one species to another.

The study of reproductive division of labor in social insects has recently expanded to include epigenetic analyses. Social insects have emerged as important models in epigenetic research for two reasons. First, they provide compelling model systems to address the grand challenge of how environmental factors ‘get under the skin’ to inﬂuence the physiological and molecular basis of complex social behavior. Second, unlike Drosophila melanogaster, all but one (Patalano et al., 2015) of the social Hymenoptera studied to date have all of the canonical enzymes for DNA methylation (Wang et al., 2006), and all have the enzymes responsible for hydroxymethylation (Cingolani et al., 2013; Wojciechowski et al., 2014; Patalano et al., 2015). Social Hymenoptera also possess fully functional histone modiﬁcation systems, which have also been implicated in the regulation of division of labor (Spannhoff et al., 2011; Dickman et al., 2013; Simola et al., 2013b, 2016).

Most of the work in this new line of study concerns the role of DNA methylation in the regulation of caste determination in honeybees. There are global decreases in DNA methylation during larval development and the specification of queen and worker morphology in the honeybee (Elango et al., 2009; Lyko et al., 2010; Foret et al., 2012; Herb et al., 2012). Moreover, RNAi knockdown of the expression of DNMT3, involved in de novo DNA methylation, caused an increase in the frequency of queen development (Kucharski et al., 2008). Within the hive, these methylation patterns appear to be controlled directly by the diet fed to the larva (Foret et al., 2012), and the presence of phytochemicals that upregulate DNMT3 (Mao et al., 2015) in worker jelly appears to be, in part, responsible for the influence of diet on caste determination. There also is evidence from other advanced eusocial species (including the ants S. invicta (Wurm et al., 2011; Glastad et al., 2014), Harpegnathos saltator, and Camponotus floridanus (Bonasio et al., 2012) and the termites Reticulitermes flavipes, Coptotermes formosanus (Glastad et al., 2013), and Zootermopsis nevadensis (Terrapon et al., 2014) to support the idea that methylation systems are involved in the regulation of caste determination. Genes involved in JH and insulin signal transduction are differentially methylated between queenand worker-destined larvae (Foret et al., 2012), and RNAi targeted to IRS and Tor can reduce global methylation while preventing queen specification (Mutti et al., 2011). These results strongly suggest that the central role that epigenetic factors play in honeybee caste determination is mediated by endocrine pathways. Moreover, differential

DNA methylation of the EGFR gene induces continuous variation in body size in C. ﬂoridanus (Alvarado et al., 2015), indicating that epigenetic and nutrient signaling pathways likely intersect in many social insects to regulate physiology, morphology, and allometry.

The importance of DNA methylation in primitively eusocial species is less clear. There are differing reports of the presence of low to moderate levels of caste-specific methylation in vespid wasps (Weiner et al., 2013; Patalano et al., 2015; Standage et al., 2016), and at least one species, Polistes canadensis, appears to lack a functional DNMT3 gene (Patalano et al., 2015; Standage et al., 2016). In addition, the primitively eusocial ant Dinoponera quadriceps does not exhibit caste-specific methylation patterns (Patalano et al., 2015), nor does the clonal raider ant Cerapachys biroi (Libbrecht et al., 2016). However, in the bumblebee, specific methylation patterns are associated with reproductive state, and decreasing the global levels of methylation via pharmacological manipulation is sufficient to increase the likelihood of egg-laying behavior (Amarasinghe et al., 2014; Sadd et al., 2015).

Epigenetic analyses in social insects are in their infancy, but hold great promise for further elucidating the regulation of all forms of division of labor, including our understanding of endocrine effects at the molecular level. In addition to the wealth of information about the organizational effects of nutrition-related endocrine signaling on reproductive division of labor in honeybees, there have also been a few studies linking these pathways to the physiological and transcriptomic changes associated with the activation of reproductive behavior in previously sterile bees that hint at endocrine inﬂuences still to be discovered. Early comparisons of brain transcriptomic state in honeybee queens, sterile workers, and workers with activated ovaries revealed dramatic differences between workers and queens (including genes associated with IIS and TOR signaling), but relatively few differences between sterile and fertile workers (Thompson et al., 2006, 2008; Grozinger et al., 2007). However, more recent studies have discovered much more robust gene expression changes between sterile and fertile workers and highlighted a potential role for metabolic, insulin, and TOR signaling in ovary activation in laying workers (Cardoen et al., 2011; Galbraith et al., 2016). Moreover, brain gene expression in reproductive workers shifts to become more queenlike (Grozinger et al., 2007), suggesting that they may, to some extent, reﬂect an ancestral state of

25 eusociality. This is supported by the ﬁnding that reproductive workers, rather than being purely selﬁsh, engage in both brood care and foraging behavior for the whole colony, as they typically do in a queenright colony (Naeger et al., 2013). The notion that reproductive workers are a kind of intermediate state between the queen and sterile workers also is supported by evidence from B. terrestris, where the shift toward queenlike gene expression in reproductive workers is even more profound (Harrison et al., 2015).

1.1.5 The Role Biogenic Amines and Neuropeptides in Reproductive Division of Labor The biogenic amines (in particular, dopamine) have repeatedly been implicated in studies of the transition from sterility to fertility in reproductive workers. In worker bumblebees, honeybees, and paper wasps (Polistes chinensis), high levels of dopamine are found in the brain during the last stage of oocyte development (Harris and Woodring, 1995; Bloch et al., 2000c; Sasaki et al., 2007). A study of queenless honeybee workers further suggests that oral treatment with dopamine leads to a higher frequency of activated ovarioles (Dombroski et al., 2003). In addition, dopamine synthesis may account for enhanced ovarian development in queenless honeybee workers treated with tyramine (Sasaki and Harano, 2007), though a direct effect of tyramine on oogenesis cannot be excluded (Sasaki and Nagao, 2002). Evidence linking dopamine to fertility and dominance hierarchies in the ants H. saltator (Penick et al., 2014) and of the genus Diacamma (Okada et al., 2015) also exists, indicating it may play a role in regulating reproductive division of labor in multiple taxa. There may also be substantial variation in the importance of dopamine across species, however, because the gonadotropic effect of dopamine has been demonstrated in some queenless ants (Diacamma sp. (Okada et al., 2015)), but not other, closely related, species (S. peetersi (Cuvillier-Hot and Lenoir, 2006)), where octopamine appears to govern fertility instead (see below).

Despite the strong relationship between dopamine and worker ovary development, the association between reproduction, caste determination, and elevated brain dopamine levels is less clear in bumblebee and honeybee queens. In honeybees, virgin queens have higher brain and hemolymph levels of dopamine, dopamine metabolites, and enzymes involved in dopamine synthesis (Sasaki et al., 2012), yet dopamine is lower in mated queens with activated ovaries than in virgin queens (Harano et al., 2005). Similarly, in bumblebees, dopamine levels are associated with oocyte development in workers but not in queens (Bloch et al., 2000c). Although

26 these observations may point to caste-related variability in the involvement of dopamine in oogenesis, it is notable that in both bee species dopamine levels were signiﬁcantly higher in queens than in workers, even those with activated ovaries. Thus, it is possible that dopamine has similar roles in queen and worker ovarian development, but other functions mask the association between ovary state and brain dopamine levels.

Brain octopamine levels are highest in dominant queenright worker bumblebees (Bloch et al., 2000c) and S. peetersi workers (Cuvillier-Hot and Lenoir, 2006). This is particularly intriguing, as increased octopamine levels (in both the hemolymph and brain) lead to enhanced arousal in several different nonsocial insect species and have also been implicated in the regulation of dominance behavior in crustaceans (Kravitz, 1988, 2000). However, the fact that dominant bumblebees with high levels of octopamine also engage in egg laying indicates that octopamine may be associated with this behavior in addition to, or independent, of, aggression. Octopaminedeﬁcient fruit ﬂies show an inhibition of egg-laying behavior that is eliminated by octopamine treatment (Monastirioti et al., 1996), hinting at a deeply conserved role for octopamine in insect egg laying behavior. This role is not universal, however, as octopamine is not associated with egg-laying behavior and ovarian development in workers of the wasp P. chinensis (Sasaki et al., 2007).

Quantitative trait loci (QTL) analyses of reproductive workers have also revealed heritable genetic components to this behavior and have implicated genes involved in biogenic amine signaling in ovary activation (Oxley et al., 2008). Studies examining the expression of some of the genes encompassed by these QTLs, including the dopamine receptor dop3 and octopamine receptor OA1, showed a negative correlation with both queen’s presence and ovary activation, although the serotonin receptor HT7R has the opposite relationship with ovary activation (Vergoz et al., 2012). In addition, inhibiting the expression of a tyramine receptor gene affected ovary size (Wang et al., 2012b). Since brain and ovary biogenic amines and their receptors are responsive to the presence or absence of the queen (Beggs et al., 2007; Vergoz et al., 2012), these results suggest that there may be a hitherto undetected direct connection between the nervous and reproductive systems that governs fertility. This is hinted at by transcriptomic analyses of sterile and reproductive honeybee workers (Cardoen et al., 2011) that revealed large differences in neuropeptide expression, including the upregulation of Allatostatin A

(which inhibits JH production in the corpora allata). Therefore, peptidomic analyses of reproductive behavior might be a fruitful avenue for future research to link neural and endocrine analyses of reproductive division of labor.

1.2.0 Endocrine Influences on Division of Labor Among Workers: Behavioral Maturation The evolution of a highly structured worker force in a social insect colony is generally seen as an evolutionary consequence of the developmental divergence between queens and workers. According to this scenario, once workers were limited to serve as helpers, their characteristics could be shaped further by natural selection acting at the level of the society to increase colony ﬁtness (West-Eberhard, 1975, 1978; Oster and Wilson, 1978; Hunt, 2012). Because division of labor increases the efﬁciency and reliability with which multiple tasks are accomplished (Oster and Wilson, 1978), some of the key colonylevel adaptations in insect societies have involved the evolution of systems of division of labor among workers.

Most studies of how hormones influence age-related division of labor among workers have been conducted with honeybees. Worker honeybees typically work inside the nest in a sequence of tasks for approximately the first 2–3 weeks of adulthood and then shift to defending the nest and foraging outside for the remainder of their 6to 8-week adult life span (Winston, 1987; Robinson, 2002). Because the most prominent activity performed by older bees is foraging, the age at onset of foraging has been used as a key developmental milestone in studies of age-related division of labor and its endocrine influence.

1.2.1 JH, Vg, and Behavioral Maturation Treatment with JH, JH analogs, or JH mimics leads to accelerated behavioral development (Robinson et al., 1989; Huang and Robinson, 1992) and an early onset of foraging behavior. These results are consistent with measurements of circulating JH titers, which are typically low in honeybees that work in the hive performing brood care (nursing) and other activities, but are high in foragers (Rutz et al., 1976; Fluri et al., 1982; Robinson, 1987b; Robinson et al., 1989; Huang et al., 1994). However, JH is not the sole regulator of foraging behavior, since actively foraging bees sampled in the late winter or early spring had low titers of JH (Huang and Robinson, 1995). Although the JH hemolymph titers of active winter foragers were higher than in bees that were not

28 foraging, they were substantially lower than foragers collected in the summer and early fall. This seasonal variation in forager JH titers suggested that an elevated JH titer may not be required for a bee to mature into a forager, an idea that was later conﬁrmed by allatectomy (Sullivan et al., 2000). Allatectomized bees could still forage, but they did so several days later than sham-operated bees, and this delay was eliminated with hormone replacement. Although few hormone replacement therapy studies have been performed in other species, JH and JH analog treatments offer remarkably consistent evidence for a similar role of JH in the regulation of behavioral maturation across the Hymenoptera, though some exceptions do exist (Table 1.1).

Age-related division of labor, while highly structured, is also quite ﬂexible (Robinson, 1992). Worker honeybees are able to respond to changing colony needs by altering or even reversing their typical patterns of behavioral maturation based on environmental and social cues such as weather, season, colony nutritional status, contact with the brood and queen, and colony demography (Huang and Robinson, 1992, 1996; Robinson, 1992; Schulz et al., 1998). For instance, foragers inhibit the maturation of younger bees in a manner reminiscent of social inhibition of reproduction in bumblebees (discussed above) and social regulation of puberty in mice (Huang and Robinson, 1999). This social inhibition is mediated by a worker-produced pheromone (Leoncini et al., 2004). Other pheromones, produced by the queen (Pankiw et al., 1998a) and brood (Ledoux et al., 2001), also inhibit JH biosynthesis and thereby delay the onset of foraging behavior (Le Conte and Hefetz, 2008). Since higher levels of brood or queen pheromones reﬂect increased colony broodrearing activity, it would be advantageous for workers to compensate by delaying the onset of foraging and increasing the duration of the broodrearing phase. There are undoubtedly more inhibitory factors that remain to be discovered in the beehive, and likely other factors that promote behavioral development as well (Pankiw, 2004).

The responsiveness of honeybees to the various environmental and social cues that inﬂuence age-related division of labor is mediated by the endocrine system. Young bees show precocious foraging in age-matched colonies that lack a normal complement of foragers and have JH titers that are comparable to those of normal-age foragers despite their youth (Robinson et al., 1989; Huang and Robinson, 1992). Similarly, young bees in colonies deﬁcient in nurses will continue to tend brood despite advancing chronological

29 age; these overage nurses continue to have low JH titers corresponding to levels in normal-age nurses (Robinson et al., 1989). Behavioral reversion, from foraging to nursing, may also occur if there are no other nurses in the colony (Page et al., 1992; Robinson et al., 1992a), and reverted nurses show correspondingly lower JH titers (Robinson, 1992; Huang and Robinson, 1996). Bee pheromones that inhibit rates of behavioral development also depress JH titers (Kaatz et al., 1992; Pankiw et al., 1998a; Le Conte et al., 2001).

Another classic role of the endocrine system is to coordinate changes in physiological and behavioral development, as in the pleiotropic effects of vertebrate steroid hormones (e.g., Pfaff, 1999; Ketterson and Nolan, 2000). JH appears to play this role in honeybee division of labor, inﬂuencing some of the striking age-related changes in the activity of exocrine glands whose secretions are vital to division of labor and colony functioning in general (Winston, 1987). For example, when workers are young and in the nursing phase of life, their hypopharyngeal glands are largest and produce some of the material that they feed to the brood; these glands shrink in size in foragers and, instead, produce a glucosidase involved in the conversion of nectar to honey. Low titers of JH or rates of JH biosynthesis are typically associated with well-developed hypopharyngeal glands, while JH treatment induces premature hypopharyngeal gland shrinkage (Huang et al., 1994).

Similarly, production of the alarm pheromones 2-heptanone and isoamyl acetate increases with age, as older bees are more defensive than younger bees (Winston, 1987), and JH analog treatment induces premature production of alarm pheromones (Robinson, 1985). Muller and Hepburn (1994) reported that JH manipulations did not affect the timing of activity for another important exocrine gland, the wax glands, that produce material used to build the honeycombs. However, wax gland activity does not change as sharply with age as do the other glands mentioned above (Muller and Hepburn, 1992, 1994). JH therefore appears to be a major factor in the connection between behavior and exocrine gland development in honeybees.

Although the proximal molecular mechanisms by which JH inﬂuences central nervous system function in adult social insects are unclear (Pandey and Bloch, 2015), it has been hypothesized that JH modulates response thresholds to stimuli (likely olfactory, tactile,

30 or auditory) that elicit the performance of specific tasks within the colony (Robinson, 1987a,b). This idea has received substantial support over time (Beshers et al., 1999; Barron and Robinson, 2008; Gove et al., 2009). Behavioral analyses have shown that honeybee workers became more responsive to alarm pheromone and less responsive to queen pheromone as they age, effects that were not mediated by changes in peripheral (antennal) chemoreceptors (Allan et al., 1987; Robinson, 1987a; Phamdelegue et al., 1993), suggesting a positive trend between JH and sensitivity to social cues in the central nervous system. Moreover, treatment with JH analog caused a premature sensitivity to alarm pheromones (Robinson, 1987a). These results are consistent with findings indicating that JH results in changes in responsiveness to behaviorally relevant stimuli in other species as well (Anton and Gadenne, 1999; Stout et al., 2002). There also is an age-related increase in responsiveness to sucrose in honeybees linked to the performance of foraging tasks (Pankiw and Page, 1999; Page and Amdam, 2007), and JH has been implicated in this process as well (Pankiw and Page, 2003). Hormonal modulation of the responsiveness to task-related stimuli thus provides a plausible explanation for how JH might influence division of labor.

JH has substantial effects on brain gene expression in the context of worker division of labor (Pandey and Bloch, 2015). Treatment with the JH analog methoprene causes marked changes in brain gene expression, inducing a transcriptomic state that closely resembles that of foragers (Whitﬁeld et al., 2006). Since JH is involved in regulating the transition to foraging behavior, this not only emphasizes the strong relationship between the brain transcriptome and neuroendocrine state but indicates that endocrine signaling likely plays a critical role in organizing gene expression across the brain, linking it directly to worker division of labor in the social insects.

Additionally, JH demonstrates extensive crosstalk with other hormonal systems to modulate gene expression related to worker division of labor (see Sections1.2.2 and 1.2.3). Perhaps the best studied of these relationships is the mutually inhibitory relationship that exists between JH and Vg (Amdam et al., 2003; Page et.al., 2012). The fact that this regulatory relationship is inverted in adults, but not in larvae, suggests that it may have been co-opted and rewired to regulate differences in adult behavior, as suggested by the reproductive ground plan hypothesis (Amdam et al., 2003). Vg levels reach their nadir in foraging bees (Amdam and Omholt, 2003) and vgRNAi administration

31 accelerates the age at onset of foraging (Nelson et al., 2007) in a manner similar to JH treatments and also results in forager-like responsiveness to ﬂoral stimuli (Scheiner et al., 2003; Amdam et al., 2006b; Nelson et al., 2007). A double knockdown of vg and the transcription factor ultraspiracle (usp, a nuclear receptor linked to JH and ecdysone signaling that itself inﬂuences behavioral maturation (Ament et al., 2012a)) had synergistic effects on gustatory responsiveness, JH titer, and foragingrelated gene expression (Wang et al., 2012a).

Selective breeding for the tendency to collect pollen over nectar has resulted in strains that are characterized by distinct behavioral traits (including precocious foraging and enhanced gustatory sensation in high pollen hoarding bees), known as pollen hoarding syndrome (Page et al., 2012). These studies have revealed that the interaction between JH, Vg, and foraging behavior is dependent on genotype, with JH levels in low pollen hoarding bees being unresponsive to vgRNAi (Nelson et al., 2007). Subsequent analyses mapping the genetic architecture underlying this differential responsiveness implicated it in ovary size, behaviors associated with the pollen hoarding syndrome, and insulin signaling, although the precise genetic variants mediating these effects have not yet been identiﬁed (Ihle et al., 2015).

1.2.2 Ecdyson and Behavioral Maturation Differential brain expression of numerous ecdysone-responsive genes in contexts related to behavioral maturation and aggression (Pandey and Bloch, 2015) is highly suggestive of a link to division of labor in honeybees. Changes in ecdysone receptor (EcR) expression and ecdysone titers are strongly coupled to downstream gene expression in pupal (Mello et al., 2014) and newly eclosed (Velarde et al., 2009) adult bees. However, there is as yet no known causal connection between ecdysteroids and worker division of labor (Hartfelder et al., 2002), and there have been relatively few studies that attempt to directly assay the behavioral and transcriptomic consequences of direct ecdysone administration. Moreover, because many of the genes downstream of EcR are associated with other hormonal pathways, particularly JH, it is possible that their expression is linked to activity in these systems instead. These factors make ecdysone’s role in worker division of labor (if any) an open question and one that deserves additional study (Pandey and Bloch, 2015).

1.2.3 Nutrition and Metabolic Factors Influencing Behavioral Maturation Nutrition is another factor with strong effects on worker– worker division of labor. Extreme nutrient deprivation has been linked to an accelerated onset of foraging behavior (Schulz et al., 1998). Moreover, bees lose lipid and protein reserves prior to the onset of foraging (Toth et al., 2005; Toth and Robinson, 2005), and treatments that cause lipid loss also lead to accelerated onset of foraging. However, it appears that this lipid loss is in fact more responsive to socially mediated factors such as pheromones rather than changes in nutrition (Ament et al., 2011a; Wheeler et al., 2015). It is therefore not surprising that insulin signaling has been implicated in behavioral maturation; IIS- related gene expression is higher in the brains and fat bodies of forager bees than in nurses (Whitﬁeld et al., 2003; Alaux et al., 2009b; Ament et al., 2011a), and direct manipulation of the TOR signaling pathway can modulate the onset of foraging behavior (Ament et al., 2008). IIS signaling may also play a central role in the stable lipid loss observed during the transition from in-hive to foraging tasks (Ament et al., 2011a). However, the fact that nurses have higher nutrient stores than foragers indicates that the classic relationship between IIS and metabolism has been reversed in the adult honeybee (Ament et al., 2011a), although not in larvae (Wheeler et al., 2006).

Changes in the expression of other genes connected to insulin/TOR pathways are also known to mediate the aggressive behavior that honeybees display when defending their colony against nest attack. This relates to age-related division of labor because it is the older, foraging-age bees that generally engage in nest defense. Both transcriptomic and metabolomic analyses indicate that decreased brain oxidative phosphorylation and increased glycolysis are strongly associated with increased aggression (Alaux et al., 2009d; Rittschof et al., 2014; Chandrasekaran et al., 2015). Moreover, pharmacological and genetic manipulations to decrease brain oxidative phosphorylation, in honeybees and D. melanogaster, respectively, increased aggression (Li-Byarlay et al., 2014), suggesting that reduced brain oxidative phosphorylation may be a key feature in regulating aggression across both insects and vertebrates (Rittschof et al., 2014). Given the central and conserved role of insulin and TOR signaling in regulating oxidative phosphorylation (Cheng et al., 2010), changes in IRS or IIS pathway expression may be at least partially mediating these changes.

The prominent role of nutrition and insulin/TOR pathways in the regulation of various components of age-related division of labor in honeybees led Wheeler et al. (2015) to perform a transcriptomic study of the pars intercerebralis (PI), the insect analog of the hypothalamus. The PI serves as one of the major neurosecretory centers of the brain as well as controls JH production in the corpora allata and prothoracicotrophic hormone in the corpora cardiaca. The PI is also the site of ILP production within the brain and appears to be responsive to insulin produced by the fat bodies (Geminard et al., 2009; Nassel et al., 2013). Since the PI is the interface between the brain and the JH and insulin axes, understanding how it incorporates these disparate signals beginning at the transcriptomic level is crucial for comprehending the role of socially and nutritionally mediated endocrine factors on division of labor.

Dissecting the inﬂuence of diet and JH on gene expression changes within the PI led to the surprising discovery that diet manipulations induced only relatively minor differences in gene expression that did not track behavioral maturation (Wheeler et al., 2015). By contrast, JH analog administration led to large-scale transcriptomic changes that closely resembled a forager-like transcriptome (Wheeler et al., 2015), as was the case for whole- brain analyses (Whitﬁeld et al., 2006). This has been interpreted as evidence that the PI’s traditional role in nutrient sensation and the regulation of appetitive state has been co-opted in honeybees to respond more strongly to socially mediated endocrine cues (Wheeler et al., 2015).

Nutritional effects are also reﬂected in transcriptomic analyses of the fat bodies, implicating this tissue in the regulation of age-related division of labor. Abdominal RNAi administration targeted to vg and the nuclear receptor usp accelerated behavioral maturation (Nelson et al., 2007; Ament et al., 2012b) and induced dramatic changes in fat body gene expression. A dual knockdown of vg and usp in the fat body has also permitted a dissection of their contributions to metabolicand insulin-related gene expression (Wang et al., 2012a). Reductions in the levels of vg and usp were found to have synergistic effects on gene expression, decreasing the levels of Ilp-1 and the foraging gene, yet increasing levels of adipokinetic hormone receptor (which mobilizes lipid stores and thus may play a role in lipid loss in foragers (Ament et al., 2011a)). This study also revealed a synergistic relationship between vg and usp and carbohydrate mobilization and starvation resistance (which were increased and decreased,

34 respectively, in bees treated with RNAi), further emphasizing the potential role of JH– insulin crosstalk in behavioral maturation.

1.2.4 The Role of Biogenic Amines and Neuropeptides in Behavioral Maturation Honeybee foragers have higher brain levels of dopamine, serotonin, and octopamine than nurses (Taylor et al., 1992; Wagener-Hulme et al., 1999). Octopamine in particular plays a causal role in regulating honeybee division of labor, as pharmacological treatments increased the likelihood of initiating foraging precociously (Schulz and Robinson, 2001). However, these effects were short-lived, suggesting that octopamine may be acting as a neuromodulator in this context. Since JH analog treatment induces high, forager-like levels of octopamine in the antennal lobes of 6and 12-day-old bees, it may be that JH inﬂuences foraging onset, in part, through its action on the octopaminergic system (Schulz et al., 2002). Additional evidence that octopamine is downstream of JH is provided by the ability of allatectomized bees to respond to octopamine treatment and initiate foraging precociously. Such a hierarchical relationship is also consistent with the differences in the timing of octopamine and JH analog treatment effects, since octopamine inﬂuences foraging much more rapidly and transiently than does JH (Robinson, 1987b; Sullivan et al., 2000; Schulz and Robinson, 2001).

This association between octopamine levels and division of labor is particularly strong in the antennal lobes, which are crucial for processing olfactory, gustatory, and tactile stimuli from the antennae (Sandoz, 2013). Using social manipulations to obtain nurse bees and foragers that are the same age, Schulz and Robinson (1999) showed that foragers had higher antennal lobe levels of octopamine than nurses (regardless of age), whereas octopamine levels in the mushroom bodies varied with bee age rather than nurse/forager status. Octopamine-based modulation of sensitivity to olfactory stimuli has been shown in laboratory assays of honeybee learning (Mercer and Menzel, 1982; Menzel and Muller, 1996) and nestmate recognition (Robinson et al., 1999). These results have led to the idea (Barron et al., 2002) that octopamine may alter behavioral state in honeybees by adjusting response thresholds to some behaviorally related stimuli in a manner similar to JH (Barron and Robinson, 2005). Moreover, since octopamine appears to be downstream of JH, it may be at least partially responsible for the changes in sensitivity induced by JH (see Section 1.2.1).

In support of this idea, Barron et al. (2002) showed that a dose of brood pheromone that elicited a minor response in untreated bees evoked a strong, positive response from octopamine-treated bees. Since brood pheromone is known to regulate foraging behavior (Pankiw et al., 1998b), this experiment effectively demonstrated that octopamine does modulate responsiveness to foraging-related stimuli. Moreover, the inﬂuence of octopamine on perception is not limited to olfactory stimuli. Octopamine increases the sensitivity of individual bees toward sucrose rewards, correspondingly enhancing the learning capabilities of young bees (Behrends and Scheiner, 2012). Levels of octopamine in the optic lobe are also connected to differences in phototactic behavior (Scheiner et al., 2014b). Since increases in gustatory responsiveness and phototaxis immediately precede the onset of foraging, these results suggest that octopamine modulates the transition to foraging by altering responsiveness in multiple sensory modalities. Tyramine (Scheiner et al., 2014a) and serotonin (Thamm et al., 2010) have also been implicated in the modulation of sensory response thresholds in the context of division of labor, indicating that this may be a common function of biogenic amines in the nervous systems of social insects.

Honeybees use a symbolic dance language to communicate the distance, direction, and quality of food sources (Von Frisch, 1967). Therefore, increases in sensitivity to foraging stimuli can also explain why octopamine treatment (and cocaine, an octopamine uptake inhibitor) causes bees to increase their assessment of the quality of food sources when communicating with nestmates (Barron and Robinson, 2005; Lehman et al., 2006; Barron et al., 2007).

Changes in the expression of neuromodulator signaling pathways have also been linked to responses to pheromones (Grozinger and Robinson, 2002; Alaux et al., 2009b) and the communication of information via a variety of behaviors (Alaux et al., 2009c). Consistent with the results presented above, brain expression of dopamine and octopamine related genes appears to play a role in behavioral maturation, especially the transition to foraging (Whitﬁeld et al., 2003; Alaux et al., 2009b).

More recent molecular results implicate these signaling pathways in determining the various specialized behaviors of individual foragers. Whereas most foragers will return

36 to the same floral source repeatedly and even persist for some time after the resource is depleted (Townsend- Mehler et al., 2010), a subset of foragers (known as scout bees) will instead constantly patrol for novel food sources and then report these back to the colony by means of the dance language. That these scouts do not return to a profitable food source has been viewed as a tendency to seek out novel stimuli. Brain transcriptomic profiling of scout bees revealed expression differences in dopamineand octopamine- related gene expression (Liang et al., 2012, 2014), paralleling findings for noveltyseeking behavior in vertebrates (Bardo et al., 1996), and pharmacological manipulation of these pathways led to increased or decreased scouting behavior in a manner predicted by the gene expression results (Liang et al., 2012). Similar findings also were obtained for another group of scouts that seek out new nest sites. Together with the fact that many of the same bees scouted in both contexts, these results indicate that biogenic amines serve to establish reliable differences in novelty-seeking behavior across different contexts and therefore underlie animal personality in honeybees.

Although their role in division of labor is not as well studied as the biogenic amines, neuropeptides have been linked to worker division of labor by several experiments (Brockmann et al., 2009; Han et al., 2015). Because many neuropeptidergic cells are found in and adjacent to higher-order processing centers of hymenopteran nervous systems and have extensive functional or structural similarities with known vertebrate peptides (Galizia and Kreissl, 2012), neuropeptides are likely an important but understudied factor in worker division of labor. For instance, neuropeptide Y and its insect orthologs neuropeptide F and short neuropeptide F (sNPF) regulate feeding behavior across taxa (Wu et al., 2003; Lee et al., 2004; Morton et al., 2006). However, in honeybees some (but not all) genes related to NPY-like signaling appear to have been co-opted to function in the context of behavioral maturation, possibly losing their ancestral functions in nutrient sensation in the process (Ament et al., 2011b). How other neuropeptides contribute to worker division of labor is an important topic for future study.

1.3.0 Endocrine Influences on Division of Labor among Workers: Morphologically Distinct Castes Among social Hymenoptera, only ants have evolved morphologically diverse workers. In most of these species, age-related division of labor and division of labor based on worker size and/or shape combine to create a diverse blend of behavioral biases

(Hölldobler and Wilson, 1990). In general, older and bigger individuals defend the colony and forage while younger and smaller individuals work in the nest. As in queen and worker caste determination, physical differences among adult workers must have their origin in pathways of development that diverge during the larval stage. Genetic variation (Hughes et al., 2003; Huang et al., 2013) can also bias the response of developing individuals to environmental factors. Extrinsic cues, including those from the social environment, must be integrated by endocrine systems to activate the appropriate developmental pathway. The endocrine basis of this type of polyphenism has been studied only in the ant genus Pheidole, which is characterized by distinctly dimorphic worker castes: soldiers and minor workers (Figure 1.3). As their name suggests, soldiers tend to be more specialized in colony defense while minor workers perform other tasks such as brood rearing. Pheidole soldiers are not simply larger minor workers; they follow different growth rules in addition to developing to a larger size (Wheeler, 1991; Holley et al., 2016).

1.3.1 JH and Worker Caste Differentiation Endocrine inﬂuences on the development of physically distinct worker castes appear similar to those for worker–queen caste determination, as described in Section 1.1.1. Current evidence is limited to treatment studies with JH analogs (Wheeler and Nijhout, 1981; Ono, 1982), which indicate that treating Pheidole larvae during the last larval instar initiates soldier development. As described earlier, queen determination in Pheidole occurs much earlier, during embryonic or early larval periods. Unfortunately, additional types of evidence for JH mediation of soldier determination, such as measurements of circulating titers and corpora allata activity and gland removal experiments, are lacking. Progress has been impeded by the small size of larvae, the lack of developmental synchronicity among larvae, and the inability to predict which larvae will develop as soldiers. Molecular tools will help make this a more tractable system.

Treatment studies have given two interesting insights into the architecture of endocrine- mediated soldier development. First, it is clear that the response to JH involves a threshold effect; treatment with a near-threshold dose produces either large minor workers or small soldiers, but never minor soldier intermediates (Wheeler and Nijhout, 1983). In contrast to the complete dimorphism that characterizes Pheidole, the most common type of polymorphic worker caste in ants features a continuous range of worker

38 sizes. In the model insects D. melanogaster and Manduca sexta, the IIS pathway, ecdysteroids, JH, and tissue-speciﬁc responses orchestrate growth rates and size thresholds for imaginal disc and overall body sizes. As a result, they are likely to underlie the speciﬁc features and evolution of the static allometries that characterize size and shape variation in all insects, including the often extreme variation seen in social Hymenoptera (Shingleton et al., 2007).

The second insight involves the interaction between JH and the putative inhibitory pheromone produced by adult soldiers. Virtually no larvae initiated soldier development when adult soldiers were experimentally induced to rear brood (Gregg, 1942; Wheeler and Nijhout, 1983). But when hormonetreated larvae were reared by soldiers, some of the larvae developed into soldiers, although the number reared by soldiers was still lower than the number that minor workers reared under the same conditions. These results led to the hypothesis that contact with soldiers causes the larvae to become less sensitive to the soldier-inducing effects of JH, which is consistent with two other lines of evidence (Wheeler and Nijhout, 1984). First, untreated minor workers reared by soldiers are slightly smaller than those reared by minor workers. In larvae of at least some holometabolous insects, the initiation of metamorphosis is inhibited if JH is present (Nijhout, 1975). If a soldier pheromone decreased the sensitivity of larvae to JH, then affected larvae would initiate metamorphosis slightly sooner and at a smaller size than those not exposed to soldiers. Second, both soldier and minor workers reared from JH-analogtreated larvae by soldiers were larger than those reared by minor workers. This can be explained by a heightened sensitivity to JH that persists after determination of either minor or soldier fate. Based on data from nonsocial insects, this heightened sensitivity is thought to lead to an extended larval period, with continued growth (Wheeler and Nijhout, 1984; Shingleton et al., 2007). Although little is known about how these systemic signals induce tissue-speciﬁc patterns of development in morphologically distinct worker castes, they have been observed to induce very speciﬁc patterns of growth and gene expression in the forewings of Pheidole soldiers but not minors (Wheeler and Nijhout, 1984; Abouheif and Wray, 2002).

Moreover, it is possible to decouple caste-speciﬁc phenotypes in different tissues by manipulating gene expression during critical periods, resulting in workers that exhibit

39 traits associated with more than one caste. By administering a histone deacetylase inhibitor (HDACi) or RNAi targeted to HDACs during a critical period in nervous system development, Simola et al. (2016) were able to alter the behavior of C. floridanus workers without changing their morphology. The authors were able to induce major workers to forage and scout, behaviors they would generally perform only rarely, without influencing their physical characteristics. Furthermore, inhibiting Creb binding protein (a histone acyltransferase) activity could rescue this effect or cause minor workers to drastically reduce the amount of time they spent foraging and scouting. Manipulating chromatin dynamics in these ways also resulted in the differential expression of numerous genes associated with endocrine signaling pathways, suggesting that these striking effects were due, in part, to the influence of hormones. Therefore, itis possible that endocrine signaling interacts with the chromatin landscape during caste determination to create a critical period for neural development and behavioral specification that is at least partially distinct from morphological development.

1.4 The Transcriptomic Architecture Linking Endocrine Signaling and Behavioral State Recent evidence from honeybees indicates that the transcriptomic architecture governing the influence of endocrine signals on age-related division of labor probably resembles pathways that drive insect development and metamorphosis. Therefore, this architecture likely consists of a series of transcriptional cascades initiated by hormone– receptor interactions and propagated by specific suites of endocrine-responsive transcription factors (TFs) depending on the physiological and behavioral state of the individual. Moreover, evidence of interactions between endocrine systems at the physiological level indicates that these transcriptional cascades are characterized by extensive ‘crosstalk,’ with certain TFs probably serving as signal integrators for multiple endocrine pathways. These TFs may then transduce context-specific signals to downstream TFs, resulting in the propagation of a signal that reflects the endocrine state of the organism. These conclusions are based on several systems biology analyses of honeybee brain transcriptomic data, reviewed below.

Chandrasekaran et al. (2011) developed a model of a honeybee brain transcriptional regulatory network (TRN) using TF and target gene coexpression data from 953 individuals over 48 distinct behavioral phenotypes. This study predicted that a small subset of TFs govern the vast majority of the observed behaviorally related differences

40 in brain gene expression. Several of these TFs, including broad and ftz-f1, are known to be critical components of endocrine signaling cascades. Moreover, by examining the individual states in which a TF’s target genes were differentially expressed, it was possible to infer whether a TF is likely to regulate a given type of behavior or response to social environment. broad was one of only four TFs predicted to regulate stable networks across all three general categories of behavior that comprise the TRN, foraging, behavioral maturation, and aggression. Since broad is known to integrate signals from multiple different endocrine systems in other insects (including JH, ecdysone, Ilp (indirectly, via deep orange), and Vg (Zhu et al., 2007; Gilbert, 2012)), it is possible that it serves to alter neurotranscriptomic proﬁles in response to changes in endocrine state to shift or reinforce behavioral state in response to the social environment. In doing so, it could selectively alter the expression of downstream TFs such as ftz-f1, which is controlled by broad during ecdysis (Gilbert, 2012) to induce contextually dependent modules of gene expression.

Bioinformatic detection of enriched cis-regulatory motifs in a gene’s upstream promoter region combined with knowledge about TF and target gene differential expression has also proven to be a useful method for inferring the regulatory circuitry underlying behavioral state (Sinha et al., 2006; Ament et al., 2012a; Rittschof et al., 2014; Khamis et al., 2015; Wheeler et al., 2015). For instance, Ament et al. (2012a) conducted a meta- analysis examining the combinatorial cis-regulatory logic underlying brain gene expression changes due to 10 different determinants of behavioral maturation (including nutritional, pheromonal, endocrine, and intracellular signaling pathways). They implicated several sets of endocrineresponsive TFs in the regulation of behavioral maturation, including several previously identiﬁed TFs, such as broad, ftz-f1, and usp. Some TFs appeared to have consistent effects on their targets across all or nearly all of these determinants, suggesting they transduce signals related to either accelerating or slowing down behavioral maturation in a context invariant manner.

Other TFs, however, were much more speciﬁc in their activity and were putatively involved in transducing information related to only a single determinant or seemed to regulate their targets differentially (i.e., activated or repressed expression) depending on the determinant being assessed. These results are broadly similar to the ﬁndings of the TRN detailed above, indicating that common themes exist in the regulation of behavioral

41 state regardless of whether transcriptomic differences are being assessed across distinct behaviors or across individual determinants of a single behavioral transition. Together, these analyses imply that some TFs are highly selective and invariant in the manner that they transduce information related to behavioral state, whereas others are much more labile and capable of inﬂuencing their targets in a context-dependent manner. However, further functional experiments are needed to validate the role of these TFs in worker division of labor, as has been done previously for usp (Ament et al., 2012b).

Given the causal connections between neuroendocrine state and worker division of labor, as well as the capacity of endocrine factors to regulate gene expression in distant and otherwise unrelated organs, it is possible that endocrine signals regulate division of labor by generating concordant patterns of expression throughout the insect. If this is the case, one would expect significant changes in the expression of downstream effector molecules associated with these hormones in response to shifts in behavioral state or discrete socially relevant experiences. As detailed previously, numerous transcriptomic studies have implicated a wide array of endocrine-responsive TFs in division of labor, including behavioral maturation (Whitfield et al., 2003; Alaux et al., 2009a; Ament et al., 2012b; Khamis et al., 2015), aggression (Alaux et al., 2009d), and various foraging specializations (Naeger et al., 2011; Liang et al., 2012, 2014; Lutz et al., 2012). Moreover, genes that were differentially expressed in both the whole brain and the PI of nurses and foragers demonstrated a high level of concordance across behavioral states and hormonal manipulations, implying that coordination in gene expression profiles occurs across distinct regions within the brain. Additionally, RNAi manipulations have revealed strong similarities between brain and fat body transcriptomic profiles (Ament et al., 2012b; Wheeler et al., 2013). This suggests that endocrine signals are activating very similar molecular pathways in the brain–fat body axis and thereby inducing systemic transcriptomic states to regulate division of labor.

This interpretation is further supported by transcriptomic analyses of glands that are associated with worker division of labor, such as the various glands involved in producing brood food (Ueno et al., 2015; Johnson and Jasper, 2016). Numerous endocrine-related genes (including kr-h1, kruppel-like factor 10, E75, and juvenile hormone esterase) exhibited concordant expression changes across eight tissue types when comparing nurse bees and foragers (i.e., each was upregulated in the same behavioral state across all tissues).

This contrasted with the proﬁles of most other regulatory genes, less involved in endocrine signaling. Together, these studies suggest that endocrine-related transcriptional cascades may result in coordinated expression changes across the organism to regulate the distinct behavioral states associated with worker division of labor.

1.5.0 Evolutionary Perspectives 1.5.1 Endocrine-Related Signatures of Selection Eusociality is thought to have evolved about a dozen times in the Hymenoptera (Bourke, 2011; Johnson et al., 2013) and at least ﬁve times in the bees alone (Bourke, 2011; Rehan and Toth, 2015). Molecular signatures of selection derived from comparative analyses of bee species exhibiting different levels of eusociality suggest that changes in aspects of endocrine signaling have featured prominently in the evolution of eusociality (Woodard et al., 2011; Hunt, 2012; Kapheim et al., 2015). Depending on the genes, species, caste (Feldmeyer et al., 2014), and behavior (Mikheyev and Linksvayer, 2015) in question, these signatures can reﬂect positive selection, purifying selection, or neutral evolution with increasing social complexity. Moreover, these signatures of selection do not always involve the same genes across all independent origins of eusociality, emphasizing the potential uniqueness of each origin (Kapheim et al., 2015). Indeed evolutionary pressures even within closely related species can result in rapid divergence in gene composition (Simola et al., 2013a). However, common themes (many of them related to endocrine regulation) have emerged that shed light on the basic processes governing the evolution of sociality.

A growing body of evidence suggests that evolutionary changes associated with the evolution of eusociality have involved rewiring of ancestral gene regulatory networks rather than evolutionary changes in protein-coding regions (Simola et al., 2013a; Kapheim et al., 2015). Kapheim et al. (2015) reported a strong increase in regulatory complexity, measured by the occurrence and diversity of cis-regulatory modules, with increased social complexity in 10 species of bees. Prominent in these results were TFs associated with endocrine signaling, including broad, Met, taiman, and tramtrack, which themselves were under negative selection. These results indicate that even though the protein-coding regions of endocrine-related genes may themselves not be subject to molecular evolution, their regulatory regions (and thereby functional outputs) are. An

43 increase in the frequency of potential DNA methylation sites in the genome also was positively correlated with increased social complexity in these species. Given the important role that methylation appears to play in caste determination and worker division of labor in many social insects, such increases in methylation could prove to be an important evolutionary route to increased phenotypic plasticity in the social insects.

This suggestion is consistent with the discovery of a parent of origin bias in the expression of numerous insulin and ecdysone-responsive genes in the honeybee (with paternal alleles being more highly expressed in both laying and sterile workers (Galbraith et al., 2016)). It is theorized that the expression of paternally-biased genes would likely enhance fecundity in workers (in opposition to genes with maternally-biased allelic expression), thereby facilitating the spread of the paternal alleles by inducing worker egg- laying. This implicates those genes with paternally biased expression in driving the transition to fertility in worker bees. Such intragenomic conﬂict is predicted by kin selection given the polyandrous nature of the honeybee (Haig, 2000; Dobata and Tsuji, 2012), again suggesting that endocrine-related gene expression is a driving force behind evolution in social insects.

Comparative studies in ants also have revealed evidence for rewiring of regulatory networks associated with increased social complexity (Simola et al., 2013a). One intriguing observation is that once regulatory networks associated with eusociality evolve, it appears they can remain in a latent state even when the species in question does not naturally exhibit the phenotype these networks are linked to (Rajakumar et al., 2012). In the Pheidole genus, some species of ants can develop an especially large, distinct soldier subcaste known as ‘supersoldiers’ that guard nest entrances. This transition is mediated by nutrition and JH, and JH can induce supersoldier development in both basal and derived Pheidole species that do not naturally form this subcaste (Rajakumar et al., 2012). This has been taken as evidence that supersoldier subcastes were present in the common ancestor of the Pheidole (and have since reappeared in different lineages independently) and that the regulatory networks underlying its manifestation were conserved during the evolution of the genus over millions of years (Rajakumar et al., 2012).

Comparative transcriptomic analyses associated with reproductive division of labor in

44 advanced eusocial wasps, ants, and bees have revealed evidence of convergent evolution, involving the concordant expression of some of the same pathways (if not speciﬁc genes) mentioned previously across different taxa to regulate caste differentiation (Berens et al., 2015). These results suggest that common ancestral systems, particularly those involved with metabolic signaling, were repeatedly co-opted to regulate reproductive division of labor across different taxa. The fact that similar sets of endocrine regulatory pathways are involved in the regulation of both reproductive and worker division of labor across a wide array of species further highlights their importance and suggests that endocrine-responsive regulatory systems may be particularly amenable to rewiring in the evolution of social behavior.

1.5.2 Speculation on the Evolution of Division of Labor: A Neuroendocrine Perspective The central role of hormones such as JH and insulin in division of labor across the social insects permits detailed examinations of how the neofunctionalization of ancestral endocrine pathways can lead to the evolution of complex sociality and social evolution in the Hymenoptera. This type of analysis beneﬁts from the fact that there are distinct levels of social complexity within the social insects, from solitary to advanced eusociality, with many species at each level. Although it is unclear whether or not there is an evolutionary trajectory that links all levels of sociality in the social insects, their existence permits heuristic speculation, as follows.

Hormones play a key role in transducing environmental stimuli to act as an interface between the organism’s internal state and its environment. Functioning in this capacity for behavior, the endocrine system interacts extensively with the nervous system. As such, hormones are pleiotropic regulators of diverse processes and may therefore be particularly amenable to selective pressures that alter their relationships with specific targets. In other words, even when the hormones themselves are not directly altered, their relationships with particular effectors can be changed in ways that give rise to novel modules of functional connectivity. As discussed in Section 1.5.1, this can be the consequence of evolutionary lability at the level of cis-regulation, resulting in the rewiring of transcriptional regulatory networks. When reversible epigenetic modifications are layered on top of these changes, this can result in novel context-specific expression patterns in response to endocrine signals. Such contextually dependent regulation can then result in the canalization of divergent phenotypes by utilizing the same endocrine

45 system.

Evolutionary analyses of endocrine regulation have illuminated our understanding of diverse biological systems, including social behavior in vertebrate societies and morphogenesis in insects (Zera et al., 2007). For instance, the ovarian ground plan hypothesis (West-Eberhard and Turillazzi, 1996) is a compelling theory for the evolution of division of labor. This hypothesis, originally articulated for wasps, posits that an ancestral hormonal axis (involving JH) governing ovarian function in solitary ancestors has been co-opted in social insects to modulate fertility and reproductive behavior in a contextdependent manner, allowing for the dramatic phenotypic plasticity observed in advanced eusocial insects. Since solitary insects cycle through speciﬁc behaviors such as nest construction, oviposition, foraging, and provisioning, West-Eberhard hypothesized that during the evolution from solitary to group living, natural variation in the behavior of some groups of individuals led to different proclivities in task performance (i.e., division of labor). Over generations, enhanced efﬁciency due to division of labor probably resulted in the exaggeration of these proclivities (whether by abiotic, social, or genetic factors) and subsequently led to the decoupling of reproductive and nonreproductive behaviors, and the formation of castes in the eusocial lineages (Hunt et al., 2007).

This framework has since been extended to create the reproductive ground plan hypothesis to explain certain aspects of worker division of labor in the honeybee (Amdam et al., 2004). According to this hypothesis, context-specific rewiring of reproductive gene regulatory networks that are involved in reproductive division of labor, especially the interplay between JH and Vg, also resulted in the evolution of worker division of labor. This view has subsequently received substantial support from both genetic and transcriptomic studies (Graham et al., 2011; Page et al., 2012; Ihle et al., 2015; but see Oldroyd and Beekman, 2008). Moreover, findings that regulatory network rewiring and increases in regulatory complexity in endocrinerelated pathways feature prominently across independently evolved eusocial lineages (Simola et al., 2013a; Kapheim et al., 2015; see Section 1.5.1) lend additional credence to the idea that ancestral reproductive gene networks may be decoupled in social insect workers to regulate division of labor. We therefore propose that a mechanistic theoretical framework of how endocrine systems might be co-opted to regulate specific social behaviors is a

46 worthwhile extension of classical evolutionary analyses.

To that end, in the following paragraphs we outline a verbal model for the evolution of division of labor based on modiﬁcations to JH signaling. The key perspective of this model is to view JH as part of a behaviorally relevant neuroendocrine system that is sensitive to social inhibition and able to orchestrate major effects on the development and behavior of both larval and adult social insects (Bloch and Grozinger, 2011). We envision this system to involve four components: (1) perception of relevant social cues; (2) a neuroendocrine response that involves differential regulation of JH production, JH degradation, or differential sensitivity to JH; (3) the speciﬁc regulatory pathways (de)activated by JH that result in phenotypic plasticity; and (4) JH-mediated effects on the development of important systems within larvae and adults, such as the timing or nature of metamorphosis, and the development of the adult reproductive and nervous systems. For purposes of brevity, we henceforth refer to this multicomponent system as the JH system.

We hypothesize that the evolution of division of labor has in many insect societies exploited (sensu Gould and Vrba (1982)) the basic insect JH system. We further hypothesize that variation in the timing of social inhibition can give rise to the many manifestations of division of labor in insect societies. This second aspect of our model is very similar to the model of Wheeler (1986), but emphasizes the importance of the social inhibition of JH during both preadult and adult stages. Endocrine mediated inhibition of conspeciﬁcs is a well-known theme in vertebrate societies (Wilson, 1975), and much is known about the endocrine correlates of such processes. Here it is important to note that social inhibition can refer to either direct communication between adult nestmates or differences in resource allocation involving trophallaxis, brood care, and other behaviors. Communication between adults and preadults via tactile stimuli or pheromone exchange appears to occur during alloparenting (Suryanarayanan et al., 2011) and maternal care (Shpigler et al., 2013) in some species, with consequences for adult size, behavior, and physiology, but the inﬂuence of communication on endocrine signaling is not well understood.

Our focus on social inhibition is inﬂuenced by the results of both treatment studies and hormone measurements in a variety of species, but with a special emphasis on

47 bumblebees and honeybees. Both species belong to the same family, the Apidae, yet (as discussed previously) differ in their social organization and the role JH plays in mediating their division of labor. The relatively close phylogenetic relationship between bumblebees and honeybees makes it easier to consider differences in endocrine regulation to be related to differences in social evolution. It has recently been suggested that honeybees and bumblebees have a common origin of eusociality (Romiguier et al., 2016), so comparative analyses of these two species could be particularly informative in determining what particular aspects of the JH system may contribute to social complexity.

Studies of many different species of insects, including those that do not live in true societies, have demonstrated that JH is part of a neuroendocrine system that is sensitive to diverse environmental factors, including social factors. Our hypothesis, that the evolution of division of labor in the insect societies relied heavily on changes in the JH system, involves five levels of social organization (Table 1.2). These five levels have been described in many theoretical accounts (e.g., Wilson, 1971; Michener, 1974; and West Eberhard, 1975) and are summarized here. Although these levels correspond to increasingly stratified and derived forms of division of labor, we do not expect that any given species’ ancestors passed through each various stage in a stepwise manner. Indeed, the sheer diversity of eusocial societies makes assigning some species to any one stage problematic, so they should not be taken as a definitive categorization, but rather as a general framework for the interactions between societal characteristics and division of labor.

1.5.2.1 Level One: Incipient Societies and Endocrine-Mediated Social Inhibition among Adults There are two scenarios for the incipient stage in the evolution of hymenopteran eusociality: unrelated adults joining together to establish a nest (West-Eberhard, 1978; Nowak et al., 2010; Bourke, 2011) or offspring remaining as adults with their mother rather than dispersing and attempting to ﬁnd a nest solitarily (Michener, 1974). In both scenarios reproductively competent adults are present together in a nest, and the likely result is socially mediated inhibition of reproduction by some individuals, leading to asymmetries in reproductive been studied in detail, the Halictid bee Megalopta genalis appears to utilize JH as both a gonadotropin (regulating fertility in queens and solitary bees) and as an indicator of social dominance (Smith et al., 2013). This ﬂexibility may

48 represent a ﬁrst step in the neofunctionalization of the JH system to mediate division of labor in reproduction. However, future studies will be needed to demonstrate a causal role of JH in M. genalis behavior and to provide a comparative framework detailing the potential role of JH in other facultatively eusocial species. The publication of a transcriptome for this species (Jones et al., 2015) should facilitate future analyses of this type.

1.5.2.2 Level Two: Preadult Endocrine-Mediated Social Inhibition In the second level, social inhibition occurs not only in the adult stage but also in the preadult stage, leading to dramatic distinctions in not only queen and worker fecundity, but in size and physiology as well. As Wheeler (1986) points out, JH- sensitive periods during the preadult stage allows for physical differences between queens and workers to develop before the adult body size is reached. Importantly, JH measurements and treatments during caste determination support the notion capacity (Michener, 1974; West-Eberhard, 1978; Sakagami and Maeta, 1987; Fewell and Page, 1999), i.e., the ovarian ground plan hypothesis. In other words, with the formation of the social group comes the potential for social inhibition of reproduction and the appearance of dominant and subordinate individuals. Perhaps the very ﬁrst proximate mechanism for this asymmetry was physical domination involving aggressive behavior and oophagy, as is still seen in extant primitively eusocial societies (Wilson, 1971). However, just as the ritualization of some forms of social behavior is thought to be adaptive (Wilson, 1975), it might have been evolutionarily advantageous for both parties, the dominant and the subordinate, to evolve a system to minimize expenditures associated with physical domination and cannibalized eggs, respectively. We hypothesize that this involved adult– adult, social inhibition of the JH system. This would stabilize an incipient division of labor and limit the reproductive activities of subordinate individuals. Based on the results reviewed above from primitively eusocial species such as sweat bees, bumblebees, and paper wasps, this may have involved the effects of endocrine inhibition on oogenesis (the relationship between JH and dominance behavior is less clear).

Facultatively eusocial insects (which exhibit ﬂexible shifts between solitary and social lifestyles) provide an ideal window into the endocrine mechanisms underlying this incipient stage of sociality. Not only do they represent some of the least derived lineages in terms of social complexity, the ability to directly compare solitary and social bees from

49 the same species allows for novel insights into the evolutionary and ecological factors that give rise to eusociality. Although few species have hat this process is a consequence of social inhibition. For instance, JH titers of worker- destined larvae are lower than queen-destined larvae in bumblebees, and the presence of a queen inhibits JH titers, an effect that can be abolished by JH analog administration (Bloch et al., 2000a; Cnaani et al., 2000a,b; Bortolotti et al., 2001). We therefore hypothesize that the social inhibition of larval JH, as seen in bumblebees, is a prerequisite for the evolution of more extensive forms of caste differentiation that are seen in advanced eusocial species. As such, the uninhibited state (that is, the queen) is the default in this species, with workers representing a derived deviation from the ancestral state. Comparisons of larval growth patterns in B. terrestris (Cnaani and Hefetz, 2001) and the finding that worker-specific genes are more derived than queen-specific ones (Feldmeyer et al., 2014) are consistent with this assertion.

1.5.2.3 Level Three: Preadult, Endocrine-Mediated Social Inhibition Enhanced by Disruptive Selection In the third level, disruptive selection leads to more elaborate mechanisms of preadult caste determination, resulting in morphologically distinct castes. With selection potentially acting on workerand queen-related traits quasiindependently, it is no longer possible to assert that the queen represents the ancestral stage. Queen inhibition of adult reproduction (generally through the release of pheromones) is still a vital component of reproductive division of labor at this stage, but the role of worker–worker inhibition through the allocation of resources during alloparenting becomes even more critical. Honeybees and other advanced eusocial species are exemplars of this stage of social elaboration. Comparing bumblebees and honeybees, it is apparent that the process of caste determination is much more extensive in the latter. In bumblebees, the differences between workers and queens are predominately in size and physiology, while in honeybees there are numerous morphological differences, such as a striking disparity in ovary development. Queen honeybee have an order of magnitude more ovarioles than workers (Winston, 1987), and this discrepancy appears to be a direct consequence of JH-mediated apoptosis during development (Capella and Hartfelder, 2002). Despite these differences, bumblebees and honeybees exhibit similar critical periods when JH can induce queen speciﬁcation (Figure 1.2). This observation suggests that factors other

50 than the timing of a JH-sensitivity are involved in the evolution of caste determination in advanced eusocial species. This is contrary to the model in Wheeler (1986), which was developed before information on caste determination in bumblebees was available (Cnaani et al., 1997, 2000a,b).

1.5.2.4 Level Four: Division of Labor among Adult Workers and Its Regulation by Endocrine- Mediated Social Inhibition The fourth level in the evolution of colony social organization is characterized by a division of labor among workers for colony maintenance and growth. The evolution of a highly structured worker force is generally seen as an evolutionary consequence of preadult mechanisms of caste determination that result in workers with dramatically curtailed reproductive options (West Eberhard, 1975; Oster and Wilson, 1978). With the reproductive potential of individuals determined primarily during preadult (JH-mediated) development in advanced eusocial species such as the honeybee, we speculate that the JH system in adults was free to become co-opted to regulate subcaste-speciﬁc properties, namely division of labor. This is consistent with the idea that once workers were limited to serve as helpers, their characteristics were shaped further by natural selection to increase colony ﬁtness (West Eberhard, 1975; Oster and Wilson, 1978).

As reviewed above, JH is known to inﬂuence worker age-related division of labor in honeybee colonies, and evidence for similar relationships exists for a number of other eusocial species including the wasps Polybia occidentalis (Odonnell and Jeanne, 1993) and Polistes dominulus (Shorter and Tibbetts, 2009), and ants Acromyrmex octospinosus (Norman and Hughes, 2016), H. saltator (Penick et al., 2011), and Pogonomyrmex californicus (Dolezal et al., 2012). Factors that slow the pace of behavioral development in adult worker honeybees, such as a high proportion of older individuals (Huang and Robinson, 1992) or pheromones produced by the queen or brood (Pankiw et al., 1998a; Le Conte et al., 2001) act to inhibit JH levels. In other words, the ability of adult worker honeybees to alter the pace of their behavioral maturation in response to changing colony conditions appears to be based on a process of social inhibition that involves JH. Similarly, Bloch and Hefetz (1999b) showed that the reproductive maturation of bumblebee workers is inhibited by the presence of older workers who also exert their effects, at least in part, on the JH system. Results from bumblebees and honeybees therefore suggest an ancient connection between the JH system’s responsiveness to

51 social environment and its role in worker and reproductive division of labor.

Due to the fact that JH acts as a gonadotropin but does not inﬂuence worker division of labor in bumblebees (and vice versa in honeybees), Cameron and Robinson (1990) proposed that JH is unlikely to function as both a gonadotropin and mediator of worker division of labor in the same species.

Moreover, since behavioral maturation is a more derived aspect of colony organization, it must represent a novel function for JH. West-Eberhard and Turillazzi (1996) countered that, because these functions are derived from a more ancestral role, arising from disruptive selection acting on reproductive and nonreproductive individuals, it is perfectly plausible that the JH system could simultaneously regulate more than one state. Evidence supporting this ‘split function’ hypothesis was found by Giray et al. (2005); in the primitively eusocial wasp P. canadensis, JH titer is correlated with both the reproductive development of queens and age-related guarding behavior in workers.

The model we present here is also more consistent with the split function hypothesis, but adds a different perspective. According to our model, the role of JH in the control of worker behavior is not a novel function at all; rather, social inhibition of JH is involved in all major stages of division of labor. Further, it appears that the reproductive ground plan of solitary bees was modiﬁed in the course of social evolution such that JH inhibits rather than stimulates vitellogenesis in accordance with the reproductive ground plan hypothesis (see Section 1.5.1). Recent evidence indicates that this inverted relationship is not unique to honeybees, but arose (presumably independently) in other advanced eusocial insects as well, such as the ants Ectatomma tuberculatum (Azevedo et al., 2016),

H. saltator (Penick et al., 2011), S. peetersi (Cuvillier-Hot et al., 2004), and L. Niger (Pamminger et al., 2016). It has further been hypothesized that this inverted relationship may be partially responsible for the dramatic differences in longevity and fecundity between workers and queens in advanced eusocial species (Corona et al., 2007; Amdam et al., 2012; Pamminger et al., 2016). However, in formicid ants, Vg has undergone one or more duplication events. The resulting paralogs, now released from selective pressures associated with Vg’s role in reproduction, have undergone neofunctionalization to develop novel inﬂuences on caste and behavior (Wurm et al.,

2011; Bonasio et al., 2012; Corona et al., 2013; Oxley et al., 2014). These studies suggest that rewiring of the JH-Vg axis is a common (but not universal) feature in the evolution of advanced eusociality.

1.5.2.5 Level Five: Division of Labor among Morphologically Distinct Adult Workers and Its Regulation by Preadult, Endocrine-Mediated Social Inhibition The ﬁfth level in the evolution of colony social organization is characterized by preadult forms of worker differentiation that result in physically distinct worker castes (Oster and Wilson, 1978). The relationship of Level Five to Level Four is very similar to the relationship of Level Two to Level One. As Wheeler (1986) points out, with queen–worker endocrine-mediated caste determination occurring earlier and earlier in preadult development, the stage is set for similar endocrine mechanisms to act later in development to give rise to diverse worker castes, such as the soldier caste in Pheidole bicarinata ants discussed in Section 1.3). This is parallel to the situation in bees, where endocrine-mediated caste determination during the preadult stage is thought to have allowed for the evolution of endocrine-mediated division of labor among workers. Of special importance to our model, the process of soldier determination in Pheidole again appears to involve social inhibition of the JH system, with the presence of soldiers inhibiting the JH-mediated differentiation process that results in soldier development (Wheeler and Nijhout, 1984). The occurrence of caste determination early in preadult development, such as in the egg stage (Suzzoni et al., 1980), can result in even greater worker–queen differences (Wheeler, 1986). For example, ﬁre ant workers have completely lost all reproductive capacity (Hölldobler and Wilson, 1990).

1.6 Concluding Remarks Our model shows how social evolution can exploit endocrine systems that are sensitive to social environment to regulate division of labor, providing a comprehensive framework for the evolution of eusociality. This model relies on the fact that JH is a pleiotropic regulator of division of labor and assumes that certain properties of the JH system (such as its sensitivity to social inhibition and connectivity to other endocrine systems) make it evolutionarily labile and readily co-opted to regulate social behavior. However, most endocrine studies of division of labor have analyzed just this hormone, and not others; is it premature to suggest that JH plays a special role in this process? As discussed previously, it is becoming increasingly apparent that other endocrine systems, such as

IIS/IRS and Vg, also play integral roles in reproductive and worker division of labor across social insect species. However, as detailed in previous sections, signaling in these pathways is inextricably tied to JH (Corona et al., 2007; Mutti et al., 2011; Amdam et al., 2012; Wang et al., 2012a,b; Libbrecht et al., 2013; Hartfelder et al., 2015).

Therefore, we believe that the general functions of JH, its well-known sensitivity to environmental stimuli, and its high level of interconnectedness with other endocrine systems make it an ideal candidate for a central role in the evolution of division of labor in the social Hymenoptera. Moreover, caste determination in the termites is also dependent on JH signaling, particularly for the speciﬁcation of the soldier caste (Miura and Scharf, 2011; Masuoka et al., 2015). The fact that JH has been utilized so extensively to organize division of labor in both Hymenoptera and insects in another order (Isoptera) suggests that some aspects of the JH system make it particularly labile and prone to modiﬁcation during social evolution, despite its crucial importance in many basal processes.

An important challenge for the future is to determine why the JH system is evolutionary labile. Newly available resources and techniques from genomic and systems biology should help. For instance, since JH has been co-opted to regulate division of labor in numerous lineages, a comparative analysis of intraspecies and interspecies differences in JH-mediated gene regulation will be an essential component for understanding how this system (and perhaps endocrine systems in general) mediates different forms of phenotypic plasticity. There are three probable sources for such plasticity: (1) context- speciﬁc rewiring of existing JH-responsive gene regulatory networks, including the JH- Vg axis; (2) the genesis of novel regulatory modules connecting JH to genes previously unregulated by this hormone; and (3) the regulation of taxon-speciﬁc genes that represent evolutionary novelties within each lineage. Understanding the identity of these effectors and the causes and consequences of their activation/expression will give important new insights into the evolution of endocrine control of social behavior.

Additionally, further attempts must be made to trace the interface between division of labor and the JH system back to its origin at the root of complex sociality. The publication of genomes and transcriptomes for the facultatively eusocial bees Eufriesia mexicana, Lasioglossum albipes, and M. genalis (Kocher et al., 2013; Jones et al., 2015; Kapheim et

54 al., 2015), and future efforts of this type, will be of great value because they will facilitate molecular interrogations of social behavior in the most ﬂexible and least derived of the social insects. The further integration of mechanistic and evolutionary analyses, coupled with the increased power of genomics and system biology, will help advance our understanding of how hormones regulate behavior in the insect societies.

Acknowledgements: I would like to thank Hagai Shpigler, Guy Bloch, Diana Wheeler and Gene Robinson for their contributions as co-authors; I would also like to acknowledge members of the Robinson lab for their input and advice. Cited work from

55 the Robinson lab was generously supported by grants from NIH, NSF, and the Simons Foundation.

1.7 Figures

Figure 1.1. Juvenile hormone (JH) acts as a gonadotropin in the bumblebee Bombus terrestris. Upper panel: measurements of rates of JH biosynthesis by the corpora allata in vitro. Middle panel: measurements of circulating titers of JH. Lower panel: measurements of ovary development. Bees were either maintained under queenless conditions in small groups in the laboratory (dashed line) or in queenright colonies (solid line) *p < 0.05. Mann–Whitney test. Reproduced from Bloch, G., Hefetz, A., Hartfelder, K., 2000b. Ecdysteroid titer, ovary status, and dominance in adult worker and queen bumblebees (Bombus terrestris). J. Insect Physiol. 46, 1033–1040, with permission from Elsevier.

Figure 1.2. Juvenile hormone (JH) titers in queens and workers of bumblebees and honeybees. Caste determination in bumblebees and honeybees is induced by the JH spike during the 3rd larval instar (L3). Whereas adult¼ bumblebees exhibit a strong, positive relationship between JH and fertility, egg-laying honeybee queens and workers exhibit far lower levels of JH than sterile foragers, indicating that JH and fertility have become decoupled in adult honeybees (see Section 1.5) L = larval stage; dashed lines are hypothetical. Based on (Strambi, et al, 1984; Hartfelder et al. 1998; Jassim et al., 2000; Bloch, et al., 2000; and Cnaani, et al, 2000).

Figure 1.3. Two periods of hormonal integration during development mediate the development of three different castes of Pheidole bicaranata ants from one totipotent genome.

1.8 Tables Table 1.1 – The Role of Juvenile Hormone in Reproductive and Worker Division of Labor Across Taxa

Social Gonadotropin Dominance Worker Citations Organization Behavior Division of Labor Wasps Polistes Primitive Yes Yes Yes (Giray, Giovanetti and canadensis West-Eberhard 2005) Polistes Primitive Yes Yes Yes (Röseler, Röseler and dominulus Strambi 1986, Roseler 1991, Shorter and Tibbetts 2009) Polistes gallicus Primitive Yes Yes ? (Bohm 1972, Barth et al. 1975, Reeve 1991, Roseler 1991) Polistes metricus Primitive Yes Yes No (Tibbetts and Sheehan 2012) Polistes smithii Primitive No No ? (Kelstrup, Hartfelder and Wossler 2015) Synoeca surinama Primitive Yes ? No (Kelstrup et al. 2014b) Polybia micans Advanced No Maybe ? (Kelstrup et al. 2014a) Polybia Advanced No ? Yes (Odonnell and Jeanne occidentalis 1993) Halictid Bees Megalopta Facultative Yes Yes N/A (Smith et al. 2013) genalis Bumblebees Bombus Primitive ? ? No (Cameron and impatiens Robinson 1990) Bombus terrestris Primitive Yes Maybe No (Bloch et al. 2000, Amsalem et al. 2014, Shpigler et al. 2014) Honeybees Apis mellifera Advanced No NA Yes (Robinson 1987, Robinson et al. 1992) Ants Lasius niger Queenless No No ? (Sommer and Holldobler 1995) Solenopsis invicta Advanced Yes NA ? (Barker 1978) Acromyrmex Advanced No NA Yes (Norman and Hughes octospinosus 2016) Pogonomyrmex Advanced No NA Yes (Dolezal et al. 2009, californicus Dolezal et al. 2012)

Table 1.2: Schematic of an endocrine-based verbal model for the evolution of social organization in hymenopteran insect societies (ants, bees, and wasps) Social complexity JH integration Solitary Environmental regulation of metamorphosis, life history, and reproduction Level 1. Simple society; similar reproductive potential Social regulation of reproduction – adult stage for all adults; division of labor linked to social rank

Level 2. Simple caste system; queens and workers differ Social regulation of reproduction – adult and in size and physiology; greater reproductive potential pre-adult for queen

Level 3. Advanced caste system; queens and workers Social regulation of reproduction – pre-adult differ in morphology; much greater reproductive caste differentiation potential for queen

Level 4. Age-related division of labor among workers Worker behavioral development (mediated by social regulation)

Level 5. Morphologically related division of labor among Worker caste differentiation (mediated by workers social regulation)

*This model proposes that the evolution of increased complexity in division of labor relates to changes in the timing of processes of social regulation that involve juvenile hormone. The model is based on knowledge that juvenile hormone systems in many solitary species of insects are sensitive to extrinsic factors. It does not imply that species or lineages of species that correspond to an advanced level of social complexity necessarily passed through the lower levels of this scheme.

References

Abouheif, E., Wray, G.A., 2002. Evolution of the gene network underlying wing polyphenism in ants.

Science 297, 249–252.

Alaux, C., Duong, N., Schneider, S.S., Southey, B.R., Rodriguez-Zas, S., Robinson, G.E., 2009a.

Modulatory communication signal performance is associated with a distinct neurogenomic state

in honey bees. PLoS One 4.

Alaux, C., Le Conte, Y., Adams, H.A., Rodriguez-Zas, S., Grozinger, C.M., Sinha, S., Robinson,

G.E., 2009b. Regulation of brain gene expression in honey bees by brood pheromone. Genes

Brain Behav. 8, 309–319.

Alaux, C., Nhi, D., Schneider, S.S., Southey, B.R., Rodriguez-Zas, S., Robinson, G.E., 2009c.

Modulatory communication signal performance is associated with a distinct neurogenomic state in

honey bees. PLoS One e6694.

Alaux, C., Sinha, S., Hasadsri, L., Hunt, G.J., Guzman-Novoa, E., DeGrandi-Hoffman, G., Uribe-Rubio,

J.L., Southey, B.R., Rodriguez-Zas, S., Robinson, G.E., 2009d. Honey bee aggression supports a link

between gene regulation and behavioral evolution. Proc. Natl. Acad. Sci. U.S.A. 106, 15400–

15405.

Allan, S.A., Slessor, K.N., Winston, M.L., King, G.G.S., 1987. The inﬂuence of age and task

specialization on the production and perception of honey bee pheromones.

J. Insect Physiol. 33, 917–922.

Alvarado, S., Rajakumar, R., Abouheif, E., Szyf, M., 2015. Epigenetic variation in the Egfr gene

generates quantitative variation in a complex trait in ants. Nat. Commun. 6.

Amarasinghe, H.E., Clayton, C.I., Mallon, E.B., 2014. Methylation and worker reproduction in the

bumble-bee (Bombus terrestris). Proc. R. Soc. B 281.

Amdam, G.V., Csondes, A., Fondrk, M.K., Page, R.E., 2006a. Complex social behaviour derived

from maternal reproductive traits. Nature 439, 76–78.

Amdam, G.V., Fennern, E., Havukainen, H., 2012. Vitellogenin in Honey Bee Behavior and Lifespan.

Springer, Netherlands, Dordrecht, pp. 17–29.

Amdam, G.V., Nilsen, K.-A., Norberg, K., Fondrk, M.K., Hartfelder, K., 2007. Variation in endocrine

signaling underlies variation in social life history. Am. Nat. 170, 37–46.

Amdam, G.V., Norberg, K., Fondrk, M.K., Page, R.E., 2004. Reproductive ground plan may mediate

colony-level selection effects on individual foraging behavior in honey bees. Proc. Natl. Acad. Sci.

U.S.A. 101, 11350–11355.

Amdam, G.V., Norberg, K., Hagen, A., Omholt, S.W., 2003. Social exploitation of vitellogenin. Proc.

Natl. Acad. Sci. U.S.A. 100, 1799–1802.

Amdam, G.V., Norberg, K., Page, R.E., Erber, J., Scheiner, R., 2006b. Downregulation of vitellogenin

gene activity increases the gustatory responsiveness of honey bee workers (Apis mellifera).

Behav. Brain Res. 169, 201–205.

Amdam, G.V., Omholt, S.W., 2003. The hive bee to forager transition in honeybee colonies: the

double repressor hypothesis. J. Theor. Biol. 223, 451–464.

Ament, S.A., Blatti, C.A., Alaux, C., Wheeler, M.M., Toth, A.L., Le Conte, Y., Hunte, G.J., Guzman-

Novoaf, E., DeGrandi-Hoffman, G., Uribe-Rubio, J.L., Amdam, G.V., Page Jr., R.E., Rodriguez-

Zas, S.L., Robinson, G.E., Sinha, S., 2012a. New meta-analysis tools reveal common

transcriptional regulatory basis for multiple determinants of behavior. Proc. Natl. Acad. Sci. U.S.A.

109, E1801–E1810.

Ament, S.A., Chan, Q.W., Wheeler, M.M., Nixon, S.E., Johnson, S.P., RodriguezZas, S.L., Foster,

L.J., Robinson, G.E., 2011a. Mechanisms of stable lipid loss in a social insect. J. Exp. Biol. 214,

3808–3821.

Ament, S.A., Corona, M., Pollock, H.S., Robinson, G.E., 2008. Insulin signaling is involved in the

regulation of worker division of labor in honey bee colonies. Proc. Natl. Acad. Sci. U.S.A. 105,

4226–4231.

Ament, S.A., Velarde, R.A., Kolodkin, M.H., Moyse, D., Robinson, G.E., 2011b. Neuropeptide Y-

like signalling and nutritionally mediated gene expression and behaviour in the honey bee. Insect

Mol. Biol. 20, 335–345.

Ament, S.A., Wang, Y., Chen, C.-C., Blatti, C.A., Hong, F., Liang, Z.S., Negre, N.,

White, K.P., Rodriguez-Zas, S.L., Mizzen, C.A., Sinha, S., Zhong, S., Robinson, G.E., 2012b. The

transcription factor ultraspiracle inﬂuences honey bee social behavior and behavior-related gene

expression. PLoS Genet. 8, 15.

Amsalem, E., Malka, O., Grozinger, C., Hefetz, A., 2014a. Exploring the role of juvenile hormone and

vitellogenin in reproduction and social behavior in bumble bees. BMC Evol. Biol. 14 (11 March

2014).

Amsalem, E., Teal, P., Grozinger, C.M., Hefetz, A., 2014b. Precocene-I inhibits juvenile hormone

biosynthesis, ovarian activation, aggression and alters sterility signal production in bumble bee

(Bombus terrestris) workers. J. Exp. Biol. 217, 3178–3185.

Anton, S., Gadenne, C., 1999. Effect of juvenile hormone on the central nervous processing of sex

pheromone in an insect. Proc. Natl. Acad. Sci. U.S.A. 96, 5764–5767.

Azevedo, D.O., de Paula, S.O., Zanuncio, J.C., Martinez, L.C., Serrao, J.E., 2016. Juvenile

hormone downregulates vitellogenin production in Ectatomma tuberculatum (Hymenoptera:

Formicidae) sterile workers. J. Exp. Biol. 219, 103–108. de Azevedo, S.V., Hartfelder, K., 2008. The insulin signaling pathway in honey bee (Apis mellifera)

caste development – differential expression of insulin-like peptides and insulin receptors in queen

and worker larvae. J. Insect Physiol. 54, 1064–1071.

Barchuk, A.R., Bitondi, M.M.G., Simoes, Z.L.P., 2002. Effects of juvenile hormone and ecdysone on

the timing of vitellogenin appearance in hemolymph of queen and worker pupae of Apis mellifera.

J. Insect Sci. (Tucson) 2, 1–9.

Barchuk, A.R., Cristino, A.S., Kucharski, R., Costa, L.F., Simoes, Z.L.P., Maleszka, R., 2007.

Molecular determinants of caste differentiation in the highly eusocial honeybee Apis mellifera.

BMC Dev. Biol. 7.

Bardo, M.T., Donohew, R.L., Harrington, N.G., 1996. Psychobiology of novelty seeking and drug

seeking behavior. Behav. Brain Res. 77, 23–43.

Barker, J.F., 1978. Neuroendocrine regulation of oocyte maturation in imported ﬁre ant

Solenopsis invicta. General Comp. Endocrinol. 35, 234–237.

Barker, J.F., 1979. Endocrine basis of wing casting and ﬂight-muscle histolysis in the

ﬁre ant Solenopsis invicta. Experientia 35, 552–554.

Barron, A.B., Maleszka, R., Vander Meer, R.K., Robinson, G.E., 2007. Octopamine modulates

honey bee dance behavior. Proc. Natl. Acad. Sci. U.S.A. 104, 1703–1707.

Barron, A.B., Robinson, G.E., 2005. Selective modulation of task performance by octopamine in

honey bee (Apis mellifera) division of labour. J. Comp. Physiol. A 191, 659–668.

Barron, A.B., Robinson, G.E., 2008. The utility of behavioral models and modules in molecular

analyses of social behavior. Genes Brain Behav. 7, 257–265.

Barron, A.B., Schulz, D.J., Robinson, G.E., 2002. Octopamine modulates responsiveness to

foraging-related stimuli in honey bees (Apis mellifera). J. Comp. Physiol. A 188, 603–610.

Barth, R.H., Lester, L.J., Sroka, P., Kessler, T., Hearn, R., 1975. Juvenile-hormone promotes

dominance behavior and ovarian development in social wasps (Polistes annularis). Experientia

31, 691–692.

Beggs, K.T., Glendining, K.A., Marechal, N.M., Vergoz, V., Nakamura, I., Slessor, K.N., Mercer, A.R.,

2007. Queen pheromone modulates brain dopamine function in worker honey bees. Proc. Natl.

Acad. Sci. U.S.A. 104.

Begna, D., Fang, Y., Feng, M., Li, J., 2011. Mitochondrial proteins differential expression during

honeybee (Apis mellifera L.) queen and worker larvae caste determination. J. Proteome Res. 10,

4263–4280.

Behrends, A., Scheiner, R., 2012. Octopamine improves learning in newly emerged bees but not in

old foragers. J. Exp. Biol. 215, 1076–1083.

Berens, A.J., Hunt, J.H., Toth, A.L., 2015. Comparative transcriptomics of convergent evolution:

different genes but conserved pathways underlie caste phenotypes across lineages of eusocial

insects. Mol. Biol. Evol. 32, 690–703.

Bernhardt, R., Waterman, M.R., 2007. Cytochrome P450 and steroid hormone biosynthesis. In: The

Ubiquitous Roles of Cytochrome P450 Proteins. John Wiley & Sons, Ltd, pp. 361–396.

Beshers, S.N., Robinson, G.E., Mittenthal, J.E., 1999. Response thresholds and division of labor in

insect colonies. Inf. Process. Soc. Insects 115–139.

Beye, M., Hasselmann, M., Fondrk, M.K., Page, R.E., Omholt, S.W., 2003. The gene csd is the

primary signal for sexual development in the honeybee and encodes an SR-type protein. Cell 114,

419–429.

Bloch, G., Borst, D.W., Huang, Z.Y., Robinson, G.E., Cnaani, J., Hefetz, A., 2000a. Juvenile

hormone titers, juvenile hormone biosynthesis, ovarian development and social environment in

Bombus terrestris. J. Insect Physiol. 46, 47–57.

Bloch, G., Borst, D.W., Huang, Z.Y., Robinson, G.E., Hefetz, A., 1996. Effects of social conditions on

Juvenile Hormone mediated reproductive development in Bombus terrestris workers. Physiol.

Entomol. 21, 257–267.

Bloch, G., Grozinger, C.M., 2011. Social molecular pathways and the evolution of bee societies.

Philos. Trans. R. Soc. B 366, 2155–2170.

Bloch, G., Hefetz, A., 1999a. Reevaluation of the role of mandibular glands in regulation of

reproduction in bumblebee colonies. J. Chem. Ecol. 25, 881–896.

Bloch, G., Hefetz, A., 1999b. Regulation of reproduction by dominant workers in bumblebee (Bombus

terrestris) queenright colonies. Behav. Ecol. Sociobiol. 45, 125–135.

Bloch, G., Hefetz, A., Hartfelder, K., 2000b. Ecdysteroid titer, ovary status, and dominance in adult

worker and queen bumble bees (Bombus terrestris). J. Insect Physiol. 46, 1033–1040.

Bloch, G., Simon, T., Robinson, G.E., Hefetz, A., 2000c. Brain biogenic amines and reproductive

dominance in bumble bees (Bombus terrestris). J. Comp. Physiol. A 186, 261–268.

Bohm, M.K., 1972. Effects of environment and juvenile-hormone on ovaries of wasp,

Polistes metricus. J. Insect Physiol. 18, 1875–1883.

Bonasio, R., Li, Q., Lian, J., Mutti, N.S., Jin, L., Zhao, H., Zhang, P., Wen, P.,

Xiang, H., Ding, Y., Jin, Z., Shen, S.S., Wang, Z., Wang, W., Wang, J., Berger, S.L., Liebig, J.,

Zhang, G., Reinberg, D., 2012. Genome-wide and castespeciﬁc DNA methylomes of the ants

Camponotus floridanus and Harpegnathos saltator. Curr. Biol. 22, 1755–1764.

Bortolotti, L., Duchateau, M.J., Sbrenna, G., 2001. Effect of juvenile hormone on caste determination

and colony processes in the bumblebee Bombus terrestris. Entomol. Exp. Appl. 101, 143–158.

Boulanger, A., Dura, J.M., 2015. Nuclear receptors and Drosophila neuronal remodeling. Biochim.

Biophys. Acta Gene Regul. Mech. 1849, 187–195.

Bourke, A.F.G., 2011. Principles of Social Evolution. xii. Oxford University Press, Oxford; New York,

p. 267.

Brent, C., Peeters, C., Dietmann, V., Crewe, R., Vargo, E., 2006. Hormonal correlates of reproductive

status in the queenless ponerine ant, Streblognathus peetersi.

J. Comp. Physiol. A 192, 315–320.

Brent, C.S., Vargo, E.L., 2003. Changes in juvenile hormone biosynthetic rate and whole body

content in maturing virgin queens of Solenopsis invicta. J. Insect Physiol. 49, 967–974.

Brockmann, A., Annangudi, S.P., Richmond, T.A., Ament, S.A., Xie, F., Southey, B.R., Rodriguez-

Zas, S.R., Robinson, G.E., Sweedler, J.V., 2009. Quantitative peptidomics reveal brain peptide

signatures of behavior. Proc. Natl. Acad. Sci. U.S.A. 106, 2383–2388.

Burns, S.N., Teal, P.E.A., Meer, R.K.V., Nation, J.L., Vogt, J.T., 2002. Identiﬁcation and action of

juvenile hormone III from sexually mature alate females of the red imported ﬁre ant, Solenopsis

invicta. J. Insect Physiol. 48, 357–365.

Buss, L.W., 1987. The Evolution of Individuality. xv. Princeton University Press, Princeton, NJ, p.

201.

Camazine, S., Deneubourg, J.-L., Franks, N.R., Sneyd, J., Theraulaz, G., Bonabeau, E., 2001. Self-

organization in biological systems. i–viii Self Organ. Biol. Syst. 1–538.

Cameron, R.C., Duncan, E.J., Dearden, P.K., 2013. Biased gene expression in early honeybee

larval development. BMC Genom. 14.

Cameron, S.A., Robinson, G.E., 1990. Juvenile-hormone does not affect division of labor in bumble

bee colonies (Hymenoptera, Apidae). Ann. Entomol. Soc. Am. 83, 626–631.

Capella, I.C.S., Hartfelder, K., 1998. Juvenile hormone effect on DNA synthesis and apoptosis in

caste-speciﬁc differentiation of the larval honey bee (Apis mellifera L.) ovary. J. Insect Physiol. 44,

385–391.

Capella, I.C.S., Hartfelder, K., 2002. Juvenile-hormone-dependent interaction of actin and spectrin

is crucial for polymorphic differentiation of the larval honey bee ovary. Cell Tissue Res. 307, 265–

272.

Cardoen, D., Wenseleers, T., Ernst, U.R., Danneels, E.L., Laget, D., De Graaf, D.C., Schoofs, L.,

Verleyen, P., 2011. Genome-wide analysis of alternative reproductive phenotypes in honeybee

workers. Mol. Ecol. 20, 4070–4084.

Cardoso, S.D., Teles, M.C., Oliveira, R.F., 2015. Neurogenomic mechanisms of social plasticity. J.

Exp. Biol. 218, 140–149.

Chandrasekaran, S., Ament, S.A., Eddy, J.A., Rodriguez-Zas, S.L., Schatz, B.R., Price, N.D.,

Robinson, G.E., 2011. Behavior-speciﬁc changes in transcriptional modules lead to distinct and

predictable neurogenomic states. Proc. Natl. Acad. Sci. U.S.A. 108, 18020–18025.

Chandrasekaran, S., Rittschof, C.C., Djukovic, D., Gu, H., Raftery, D., Price, N.D., Robinson, G.E.,

2015. Aggression is associated with aerobic glycolysis in the honey bee brain. Genes Brain

Behav. 14, 158–166.

Chen, M.E., Lewis, D.K., Keeley, L.L., Pietrantonio, P.V., 2004. cDNA cloning and transcriptional

regulation of the vitellogenin receptor from the imported ﬁre ant, Solenopsis invicta Buren

(Hymenoptera : Formicidae). Insect Mol. Biol. 13, 195–204. Chen, X., Hu, Y., Zheng, H., Cao, L.,

Niu, D., Yu, D., Sun, Y., Hu, S., Hu, F., 2012. Transcriptome comparison between honey bee

queenand worker-destined larvae. Insect Biochem. Mol. Biol. 42, 665–673.

Cheng, Z., Tseng, Y., White, M.F., 2010. Insulin signaling meets mitochondria in metabolism.

Trends Endocrinol. Metab. 21, 589–598.

Choe, J.C., Crespi, B.J., 1997. The Evolution of Social Behavior in Insects and Arachnids.

Cambridge University Press, Cambridge, p. 541.

Cingolani, P., Cao, X., Khetani, R.S., Chen, C., Coon, M., Sammak, A., Bollig-Fischer, A., Land, S.,

Huang, Y., Hudson, M.E., Garﬁnkel, M.D., Zhong, S., Robinson, G.E., Ruden, D.M., 2013. Intronic

non-CG DNA hydroxymethylation and alternative mRNA splicing in honey bees. BMC Genom. 14

(30 September 2013). Cnaani, J., Borst, D.W., Huang, Z.Y., Robinson, G.E., Hefetz, A., 1997.

Caste determination in Bombus terrestris: differences in development and rates of JH

biosynthesis between queen and worker larvae. J. Insect Physiol. 43, 373–381.

Cnaani, J., Hefetz, A., 2001. Are queen Bombus terrestris giant workers or are workers dwarf

queens? Solving the ‘chicken and egg’ problem in a bumblebee species. Naturwissenschaften

88, 85–87.

Cnaani, J., Robinson, G.E., Bloch, G., Borst, D., Hefetz, A., 2000a. The effect of queen-worker

conﬂict on caste determination in the bumblebee Bombus terrestris. Behav. Ecol. Sociobiol. 47,

346–352.

Cnaani, J., Robinson, G.E., Hefetz, A., 2000b. The critical period for caste determination in Bombus

terrestris and its juvenile hormone correlates. J. Comp. Physiol. A. 186, 1089–1094.

Corona, M., Libbrecht, R., Wurm, Y., Riba-Grognuz, O., Studer, R.A., Keller, L., 2013. Vitellogenin

underwent subfunctionalization to acquire caste and behavioral speciﬁc expression in the harvester

ant Pogonomyrmex barbatus. PLoS Genet. 9, e1003730.

Corona, M., Velarde, R.A., Remolina, S., Moran-Lauter, A., Wang, Y., Hughes, K.A., Robinson, G.E.,

2007. Vitellogenin, juvenile hormone, insulin signaling, and queen honey bee longevity. Proc. Natl.

Acad. Sci. U.S.A. 104, 7128–7133.

Cristino, A.S., Nunes, F.M.F., Lobo, C.H., Bitondi, M.M.G., Simoes, Z.L.P., Costa, L.D., Lattorff,

H.M.G., Moritz, R.F.A., Evans, J.D., Hartfelder, K., 2006. Caste development and reproduction: a

genome-wide analysis of hallmarks of insect eusociality. Insect Mol. Biol. 15, 703–714.

Crozier, R.H., Pamilo, P., 1996. Evolution of Social Insects Colonies: Sex Allocation and Kin

Selection. viii. Oxford University Press, Oxford, p. 306.

Cusson, M., McNeil, J.N., 1989. Involvement of juvenile hormone in the regulation of pheromone

release activities in a moth. Science 243, 210–211.

Cuvillier-Hot, V., Cobb, M., Malosse, C., Peeters, C., 2001. Sex, age and ovarian activity affect

cuticular hydrocarbons in Diacamma ceylonense, a queenless ant.

J. Insect Physiol. 47, 485–493.

Cuvillier-Hot, V., Lenoir, A., 2006. Biogenic amine levels, reproduction and social dominance in the

queenless ant Streblognathus peetersi. Naturwissenschaften 93, 149–153.

Cuvillier-Hot, V., Lenoir, A., Peeters, C., 2004. Reproductive monopoly enforced by sterile police

workers in a queenless ant. Behav. Ecol. 15, 970–975.

Daerr, E., 1978. Effects of allatectomy on the mating ﬂight and egg laying of newly emerged queen

honeybees. Mittl. Dtsch. Ges. Allg. Angew. Entomol. 1, 317–319.

De Loof, A., Baggerman, G., Breuer, M., Claeys, I., Cerstiaens, A., Clynen, E., Janssen, T., Schoofs,

L., Broeck, J.V., 2001. Gonadotropins in insects: an overview. Archiv. Insect Biochem. Physiol.

47, 129–138.

Dickman, M.J., Kucharski, R., Maleszka, R., Hurd, P.J., 2013. Extensive histone posttranslational

modiﬁcation in honey bees. Insect Biochem. Mol. Biol. 43, 125–137. Dobata, S., Tsuji, K., 2012.

Intragenomic conﬂict over queen determination favours genomic imprinting in eusocial

Hymenoptera. Proc. R. Soc. B 279, 2553–2560.

Dolezal, A.G., Brent, C.S., Gadau, J., Hoelldobler, B., Amdam, G.V., 2009. Endocrine physiology of

the division of labour in Pogonomyrmex californicus founding queens. Anim. Behav. 77, 1005–

1010.

Dolezal, A.G., Brent, C.S., Hoelldobler, B., Amdam, G.V., 2012. Worker division of labor and

endocrine physiology are associated in the harvester ant, Pogonomyrmex californicus. J. Exp. Biol.

215, 454–460.

Dombroski, T.C.D., Simoes, Z.L.P., Bitondi, M.M.G., 2003. Dietary dopamine causes ovary

activation in queenless Apis mellifera workers. Apidologie 34, 281–289. van Doorn, A., 1987. Investigations into the regulation of dominance behavior and of the division of

labor in bumblebee colonies (Bombus terrestris). Neth. J. Zool. 37, 255–276. van Doorn, A., 1989. Factors inﬂuencing dominance behavior in queenless bumblebee workers

(Bombus terrestris). Physiol. Entomol. 14, 211–221.

Duchateau, M.J., Velthuis, H.H.W., 1988. Development and reproductive strategies in bombus

colonies. Behaviour 107, 186–207.

Duffy, J.E., 1996. Eusociality in a coral-reef shrimp. Nature 381, 512–514.

Elango, N., Hunt, B.G., Goodisman, M.A.D., Yi, S.V., 2009. DNA methylation is widespread and

associated with differential gene expression in castes of the honeybee, Apis mellifera. Proc. Natl.

Acad. Sci. U.S.A. 106, 11206–11211.

Elekonich, M.M., Robinson, G.E., 2000. Organizational and activational effects of hormones on

insect behavior. J. Insect Physiol. 46, 1509–1515.

Engels, W., Kaatz, H., Zillikens, A., Paulino Simoes, Z.L., Trube, A., Braun, R., Dittrich, F., 1990.

Honey bee reproduction: vitellogenin and caste-speciﬁc regulation of fertility. Adv. Invertebr.

Reprod. 5, 495–502.

Engels, W., Ramamurty, P.S., 1976. Initiation of oogenesis in allatectomized virgin honey bee

queens by carbon-dioxide treatment. J. Insect Physiol. 22, 1427–1432. Evans, J.D., Wheeler,

D.E., 2001. Expression proﬁles during honeybee caste determination. Genome Biol. 2.

RESEARCH0001.

Ewards, D.H., Kravitz, E.A., 1997. Serotonin, social status and aggression. Curr. Opin. Neurobiol.

7, 812–819.

Fahrbach, S.E., Giray, T., Robinson, G.E., 1995. Volume changes in the mushroom bodies of adult

honey bee queens. Neurobiol. Learn Mem. 63, 181–191.

Feldlaufer, M.F., Herbert, E.W., Svoboda, J.A., Thompson, M.J., Lusby, W.R., 1985. Makisterone-

A the major ecdysteroid from the pupa of the honey bee, Apis mellifera. Insect Biochem. 15, 597–

600.

Feldlaufer, M.F., Svoboda, J.A., Herbert, E.W., 1986. Makisterone-A and 24-Methylenecholesterol

from the ovaries of the honey-bee, Apis mellifera L. Experientia 42, 200–201.

Feldmeyer, B., Elsner, D., Foitzik, S., 2014. Gene expression patterns associated with caste and

reproductive status in ants: worker-speciﬁc genes are more derived than queen-speciﬁc ones. Mol.

Ecol. 23, 151–161.

Fewell, J.H., Page, R.E., 1999. The emergence of division of labour in forced associations of

normally solitary ant queens. Evol. Ecol. Res. 1, 537–548.

Fletcher, D.J.C., Blum, M.S., 1981. Pheromonal control of dealation and oogenesis in virgin queen

ﬁre ants. Science 212, 73–75.

Fletcher, D.J.C., Ross, K.G., 1985. Regulation of reproduction in eusocial Hymenoptera. Annu. Rev.

Entomol. 30, 319–343.

Fluri, P., Luscher, M., Wille, H., Gerig, L., 1982. Changes in weight of the pharyngeal gland and

hemolymph titers of juvenile-hormone, protein and vitellogenin in worker honey bees. J. Insect

Physiol. 28, 61–68.

Foret, S., Kucharski, R., Pellegrini, M., Feng, S., Jacobsen, S.E., Robinson, G.E., Maleszka, R.,

2012. DNA methylation dynamics, metabolic ﬂuxes, gene splicing, and alternative phenotypes in

honey bees. Proc. Natl. Acad. Sci. U.S.A. 109, 4968–4973.

Gadagkar, R., 1991. Belonogaster, Mischocyttarus, Parapolybia, and independentfounding

Ropalidia. In: The Social Biology of Wasps, pp. 149–190.

Gadau, J., Fewell, J., 2009. Organization of Insect Societies: From Genome to Sociocomplexity. x.

Harvard University Press, Cambridge, Mass., p. 617

Galbraith, D.A., Kocher, S.D., Glenn, T., Albert, I., Hunt, G.J., Strassmann, J.E., Queller, D.C.,

Grozinger, C.M., 2016. Testing the kinship theory of intragenomic conﬂict in honey bees (Apis

mellifera). Proc. Natl. Acad. Sci. U.S.A. 113, 1020–1025.

Galizia, C.G., Kreissl, S., 2012. Neuropeptides in Honey Bees. Springer, Netherlands, Dordrecht, pp.

211–226.

Geminard, C., Rulifson, E.J., Leopold, P., 2009. Remote control of insulin secretion by fat cells in

Drosophila. Cell Metab. 10, 199–207.

Geva, S., Hartfelder, K., Bloch, G., 2005. Reproductive division of labor, dominance, and

ecdysteroid levels in hemolymph and ovary of the bumble bee Bombus terrestris. J. Insect

Physiol. 51, 811–823.

Gilbert, L.I., 2012. Insect endocrinology. online resource (x, 577p.). In: Book Insect Endocrinology.

Academic Press, City, p. 1.

Giray, T., Giovanetti, M., West-Eberhard, M.J., 2005. Juvenile hormone, reproduction, and worker

behavior in the neotropical social wasp Polistes canadensis. Proc. Natl. Acad. Sci. U.S.A. 102,

3330–3335.

Glastad, K.M., Hunt, B.G., Goodisman, M.A.D., 2013. Evidence of a conserved functional role for

DNA methylation in termites. Insect Mol. Biol. 22, 143–154.

Glastad, K.M., Hunt, B.G., Yi, S.V., Goodisman, M.A.D., 2014. Epigenetic inheritance and genome

regulation: is DNA methylation linked to ploidy in haplodiploid insects? Proc. R. Soc. B. Sci. 281.

Goodman, W.G., Granger, N.A., Gilbert, L.I., Iatrou, K., Gill, S.S., 2005. The juvenile hormones. In:

Comprehensive Molecular Insect Science. Endocrinology, vol. 3, pp. 319–408.

Gould, S.J., Vrba, E.S., 1982. Exaptation a missing term in the science of form. Paleobiology 8, 4–

15.

Gove, R., Hayworth, M., Chhetri, M., Rueppell, O., 2009. Division of labour and social insect colony

performance in relation to task and mating number under two alternative response threshold

models. Insectes Soc. 56, 319–331.

Graham, A.M., Munday, M.D., Kaftanoglu, O., Page Jr., R.E., Amdam, G.V., Rueppell, O., 2011.

Support for the reproductive ground plan hypothesis of social evolution and major QTL for ovary

traits of Africanized worker honey bees (Apis mellifera L.). BMC Evol. Biol. 11.

Gregg, R.E., 1942. The origin of castes in ants with special reference to Pheidole morrisi Forel.

Ecology 23, 295–308.

Grozinger, C.M., Fan, Y.L., Hoover, S.E.R., Winston, M.L., 2007. Genome-wide analysis reveals

differences in brain gene expression patterns associated with caste and reproductive status in

honey bees (Apis mellifera). Mol. Ecol. 16, 4837–4848.

Grozinger, C.M., Robinson, G.E., 2002. Microarray analysis of pheromone-mediated gene

expression in the honey bee brain. Integr. Comp. Biol. 42, 1237.

Grozinger, C.M., Sharabash, N.M., Whitﬁeld, C.W., Robinson, G.E., 2003. Pheromonemediated gene

expression in the honey bee brain. Proc. Natl. Acad. Sci. U.S.A. 100, 14519–14525.

Guidugli, K.R., Nascimento, A.M., Amdam, G.V., Barchuk, A.R., Omholt, S., Simoes, Z.L.P.,

Hartfelder, K., 2005. Vitellogenin regulates hormonal dynamics in the worker caste of a eusocial

insect. FEBS Lett. 579, 4961–4965.

Hagedorn, H.H., 1983. The role of ecdysteroids in the adult insect. Invertebr. Endocrinol. 1, 271–

304.

Hagedorn, H.H., 1985. 7. The role of ecdysteroids in reproduction. In: Comprehensive Insect

Physiology, Biochemistry and Pharmacology. Endocrinology 2, vol. 8, pp. 205–262.

Haig, D., 2000. The kinship theory of genomic imprinting. Annu. Rev. Ecol. Syst. 31, 9–32.

Han, B., Fang, Y., Feng, M., Hu, H., Qi, Y., Huo, X., Meng, L., Wu, B., Li, J., 2015.

Quantitative neuropeptidome analysis reveals neuropeptides are correlated with social behavior

regulation of the honeybee workers. J. Proteome Res. 14, 4382–4393.

Harano, K., Sasaki, K., Nagao, T., 2005. Depression of brain dopamine and its metabolite after

mating in European honeybee (Apis mellifera) queens. Naturwissenschaften 92, 310–313.

Harris, J.W., Woodring, J., 1995. Elevated brain dopamine levels associated with ovary development

in queenless worker honey-bees (Apis mellifera L.). Comp. Biochem. Physiol. C 111, 271–279.

Harrison, M.C., Hammond, R.L., Mallon, E.B., 2015. Reproductive workers show queenlike gene

expression in an intermediately eusocial insect, the buff-tailed bumble bee Bombus terrestris. Mol.

Ecol. 24, 3043–3063.

Hartfelder, K., Bitondi, M.M.G., Santana, W.C., Simoes, Z.L.P., 2002. Ecdysteroid titer and

reproduction in queens and workers of the honey bee and of a stingless bee: loss of ecdysteroid

function at increasing levels of sociality? Insect Biochem. Mol. Biol. 32, 211–216.

Hartfelder, K., Engels, W., 1998. Social insect polymorphism: hormonal regulation of plasticity in

development and reproduction in the honeybee. Curr. Top. Dev. Biol. 40, 45–77.

Hartfelder, K., Guidugli-Lazzarini, K.R., Cervoni, M.S., Santos, D.E., Humann, F.C., 2015. Old

threads make new tapestry-rewiring of signalling pathways underlies caste phenotypic plasticity

in the honey bee, Apis mellifera L. Adv. Insect Physiol. 48, 1–36.

Herb, B.R., Wolschin, F., Hansen, K.D., Aryee, M.J., Langmead, B., Irizarry, R., Amdam, G.V.,

Feinberg, A.P., 2012. Reversible switching between epigenetic states in honeybee behavioral

subcastes. Nat. Neurosci. 15, 1371–1373.

Hoelldobler, B., Wilson, E.O., 2008. The Superorganism: the Beauty, Elegance, and Strangeness of

Insect Societies, vol. i–xxi. W.W. Norton and Company, pp. 1–522.

Hölldobler, B., Wilson, E.O., 1990. The Ants. xii. Belknap Press of Harvard University Press,

Cambridge, Mass, 732 pages, 24 pages of plates pp.

Holley, J.-A.C., Moreau, C.S., Laird, J.G., Suarez, A.V., 2016. Subcaste-speciﬁc evolution of head

size in the ant genus Pheidole. Biol. J. Linn. Soc. 118 (3), 472–485.

Holman, L., Jorgensen, C.G., Nielsen, J., d’Ettorre, P., 2010. Identiﬁcation of an ant queen

pheromone regulating worker sterility. Proc. R. Soc. B 277, 3793–3800.

Huang, M.H., Wheeler, D.E., Fjerdingstad, E.J., 2013. Mating system evolution and worker caste

diversity in Pheidole ants. Mol. Ecol. 22, 1998–2010.

Huang, Z.Y., Robinson, G.E., 1992. Honeybee colony integration: worker–worker interactions

mediate hormonally regulated plasticity in division-of-labor. Proc. Natl. Acad. Sci. U.S.A. 89,

11726–11729.

Huang, Z.Y., Robinson, G.E., 1995. Seasonal-changes in juvenile-hormone titers and rates of

biosynthesis in honey-bees. J. Comp. Physiol. B 165, 18–28.

Huang, Z.Y., Robinson, G.E., 1996. Regulation of honey bee division of labor by colony age

demography. Behav. Ecol. Sociobiol. 39, 147–158.

Huang, Z.Y., Robinson, G.E., 1999. Social control of division of labor in honey bee colonies. Inf.

Process. Soc. Insects 165–186.

Huang, Z.Y., Robinson, G.E., Borst, D.W., 1994. Physiological correlates of division-oflabor among

similarly aged honey-bees. J. Comp. Physiol. A 174, 731–739.

Hughes, W.O.H., Sumner, S., Van Borm, S., Boomsma, J.J., 2003. Worker caste polymorphism

has a genetic basis in Acromyrmex leaf-cutting ants. Proc. Natl. Acad. Sci. U.S.A. 100, 9394–

9397.

Hunt, G.J., Amdam, G.V., Schlipalius, D., Emore, C., Sardesai, N., Williams, C.E., Rueppell, O.,

Guzman-Novoa, E., Arechavaleta-Velasco, M., Chandra, S., Fondrk, M.K., Beye, M., Page, R.E.,

2007. Behavioral genomics of honeybee foraging and nest defense. Naturwissenschaften 94,

247–267.

Hunt, J.H., 2012. A conceptual model for the origin of worker behaviour and adaptation of

eusociality. J. Evol. Biol. 25.

Ihle, K.E., Rueppell, O., Huang, Z.Y., Wang, Y., Fondrk, M.K., Page Jr., R.E., Amdam, G.V., 2015.

Genetic architecture of a hormonal response to gene knockdown in honey bees. J. Hered. 106,

155–165.

Imboden, H., Luescher, M., 1975. Allatectomy in adult worker bees Apis mellifica.

Rev. Suisse Zool. 82, 694–698.

Imboden, H., Wille, H., Gerig, L., Luescher, M., 1976. Vitellogenin synthesis in the worker honey

bee Apis mellifera and its dependence on juvenile hormone. Rev. Suisse Zool. 83, 928–933.

Jarau, S., van Veen, J.W., Twele, R., Reichle, C., Herrera Gonzales, E., Aguilar, I., Francke, W.,

Ayasse, M., 2010. Workers make the queens in Melipona bees: identiﬁcation of geraniol as a

caste determining compound from labial glands of nurse bees. J. Chem. Ecol. 36, 565–569.

Jarvis, J.U.M., 1981. Eusociality in a mammal cooperative breeding in naked molerat colonies.

Science 212, 571–573.

Jay, S.C., Jay, D.H., 1976. Effect of various types of brood comb on ovary development of worker-

honeybees. Can. J. Zool. Revue Can. Zool. 54, 1724–1726.

Jeanne, R.L., 1991a. Polyethism. In: The Social Biology of Wasps, pp. 389–425.± Jeanne, R.L.,

1991b. The swarm-founding polistinae. In: Ross, K.G., Matthews, R.W. (Eds.), The Social Biology

of Wasps, vol. xvii 678p.. Cornell University Press, Ithaca, New York, USA; London, England, UK,

pp. 191–231. Illus. Maps.

Johnson, B.R., Borowiec, M.L., Chiu, J.C., Lee, E.K., Atallah, J., Ward, P.S., 2013. Phylogenomics

resolves evolutionary relationships among ants, Bees, and wasps. Curr. Biol. 23, 2058–2062.

Johnson, B.R., Jasper, W.C., 2016. Complex Patterns of Differential Expression in Candidate

Master Regulatory Genes for Social Behavior in Honey Bees, pp. 1–11.

Jones, B.M., Wcislo, W.T., Robinson, G.E., 2015. Developmental transcriptome for a facultatively

eusocial bee, Megalopta genalis. G3-Genes Genomes Genet. 5,

2127–2135.

Kaatz, H.H., 1987. Changes in ecdysteroid and juvenile hormone hemolymph titers of honey bee queens

during pupal and adult development. Chem. Biol. Soc. Insects 312.

Kaatz, H.H., Hildebrandt, H., Engels, W., 1992. Primer effect of queen pheromone on juvenile-

hormone biosynthesis in adult worker honey-bees. J. Comp. Physiol. B 162, 588–592.

Kamakura, M., 2011. Royalactin induces queen differentiation in honeybees. Nature 473, 478–483.

Kapheim, K.M., Pan, H., Li, C., Salzberg, S.L., Puiu, D., Magoc, T., Robertson, H.M., Hudson, M.E.,

Venkat, A., Fischman, B.J., Hernandez, A., Yandell, M., Ence, D., Holt, C., Yocum, G.D., Kemp,

W.P., Bosch, J., Waterhouse, R.M., Zdobnov, E.M., Stolle, E., Kraus, F.B., Helbing, S., Moritz,

R.F.A., Glastad, K.M., Hunt, B.G., Goodisman, M.A.D., Hauser, F., Grimmelikhuijzen, C.J.P.,

Pinheiro, D.G., Franco Nunes, F.M., Miranda Soares, M.P., Tanaka, E.D., Paulino Simoes, Z.L.,

Hartfelder, K., Evans, J.D., Barribeau, S.M., Johnson, R.M., Massey, J.H., Southey, B.R.,

Hasselmann, M., Hamacher, D., Biewer, M., Kent, C.F., Zayed, A., Blatti III, C., Sinha, S.,

Johnston, J.S., Hanrahan, S.J., Kocher, S.D., Wang, J., Robinson, G.E., Zhang, G., 2015.

Genomic signatures of evolutionary transitions from solitary to group living. Science 348, 1139–

1143.

Kearney, G.P., Toom, P.M., Blomquist, G.J., 1977. Induction of dealation in virgin female Solenopsis

invicta (Hymenoptera-Formicidae) with juvenile hormones. Ann. Entomol. Soc. Am. 70, 699–701.

Keller, L., 1999. Levels of Selection in Evolution. xii. Princeton University Press, Princeton, N.J, p.

318.

Keller, L., Nonacs, P., 1993. The role of queen pheromones in social insects queen control or queen

signal. Anim. Behav. 45, 787–794.

Kelstrup, H.C., Hartfelder, K., Nascimento, F.S., Riddiford, L.M., 2014a. Reproductive status,

endocrine physiology and chemical signaling in the neotropical, swarmfounding eusocial wasp

Polybia micans. J. Exp. Biol. 217, 2399–2410.

Kelstrup, H.C., Hartfelder, K., Nascimento, F.S., Riddiford, L.M., 2014b. The role of juvenile

hormone in dominance behavior, reproduction and cuticular pheromone signaling in the caste-

ﬂexible epiponine wasp, Synoeca surinama. Front. Zool. 11.

Kelstrup, H.C., Hartfelder, K., Wossler, T.C., 2015. Polistes smithii vs. Polistes dominula: the

contrasting endocrinology and epicuticular signaling of sympatric paper wasps in the ﬁeld. Behav.

Ecol. Sociobiol. 69, 2043–2058.

Kerr, W.E., 1969. Some aspects of the evolution of social bees (Apidae). Evol. Biol. 3, 119–175.

Ketterson, E.D., Nolan, V., 2000. Adaptation, exaptation, and constraint: a hormonal perspective

(vol 154, pg S4, 1999). Am. Nat. 155, U7.

Khamis, A.M., Hamilton, A.R., Medvedeva, Y.A., Alam, T., Alam, I., Essack, M., Umylny, B.,

Jankovic, B.R., Naeger, N.L., Suzuki, M., Harbers, M., Robinson, G.E., Bajic, V.B., 2015. Insights

into the transcriptional architecture of behavioral plasticity in the honey bee Apis mellifera. Sci.

Rep. 5, 11136.

Koch, V., Nissen, I., Schmitt, B.D., Beye, M., 2014. Independent evolutionary origin of fem

paralogous genes and complementary sex determination in hymenopteran insects. PLoS One 9.

Kocher, S.D., Li, C., Yang, W., Tan, H., Yi, S.V., Yang, X., Hoekstra, H.E., Zhang, G., Pierce, N.E.,

Yu, D.W., 2013. The draft genome of a socially polymorphic halictid bee, Lasioglossum albipes.

Genome Biol. 14.

Kravitz, E.A., 1988. Hormonal-control of behavior amines and the biasing of behavioral output in

lobsters. Science 241, 1775–1781.

Kravitz, E.A., 2000. Serotonin and aggression: insights gained from a lobster model system and

speculations on the role of amine neurons in a complex behavior.

J. Comp. Physiol. A 186, 221–238.

Kropacova, S., Haslbachova, H., 1971. The inﬂuence of queenlessness and of unsealed brood on

the development of the ovaries in worker honeybees. J. Apic. Res. 10, 57–61.

Kucharski, R., Maleszka, J., Foret, S., Maleszka, R., 2008. Nutritional control of reproductive status

in honeybees via DNA methylation. Science 319, 1827–1830. Laidlaw, H.H., Page, R.E., 1997.

Queen Rearing and Bee Breeding. Wicwas Press, Cheshire CT, USA.

Larrere, M., Couillaud, F., 1993. Role of juvenile-hormone biosynthesis in dominance status and

reproduction of the bumblebee, Bombus terrestris. Behav. Ecol. Sociobiol. 33.

Le Conte, Y., Hefetz, A., 2008. Primer pheromones in social hymenoptera. Annu. Rev.

Entomol. 53, 523–542.

Le Conte, Y., Mohammedi, A., Robinson, G.E., 2001. Primer effects of a brood pheromone on

honeybee behavioural development. Proc. R. Soc. Lond. Ser. B 268, 163–168.

Ledoux, M.N., Winston, M.L., Higo, H., Keeling, C.I., Slessor, K.N., Le Conte, Y., 2001. Queen

pheromonal factors inﬂuencing comb construction by simulated honey bee (Apis mellifera L.)

swarms. Insectes Soc. 48, 14–20.

Lee, K.S., You, K.H., Choo, J.K., Han, Y.M., Yu, K., 2004. Drosophila short neuropeptide F regulates

food intake and body size. J. Biol. Chem. 279, 50781–50789. Lehman, H.K., Schulz, D.J., Barron,

A.B., Wraight, L., Hardison, C., Whitney, S., Takeuchi, H., Paul, R.K., Robinson, G.E., 2006.

Division of labor in the honey bee (Apis mellifera): the role of tyramine beta-hydroxylase. J. Exp. Biol.

209, 2774–2784.

Lengyel, F., Westerlund, S.A., Kaib, M., 2007. Juvenile hormone III inﬂuences taskspeciﬁc cuticular

hydrocarbon proﬁle changes in the ant Myrmicaria eumenoides. J. Chem. Ecol. 33, 167–181.

Leoncini, I., Le Conte, Y., Costagliola, G., Plettner, E., Toth, A.L., Wang, M.W., Huang, Z., Becard,

J.M., Crauser, D., Slessor, K.N., Robinson, G.E., 2004. Regulation of behavioral maturation by a

primer pheromone produced by adult worker honey bees. Proc. Natl. Acad. Sci. U.S.A. 101,

17559–17564.

Li-Byarlay, H., Rittschof, C.C., Massey, J.H., Pittendrigh, B.R., Robinson, G.E., 2014. Socially

responsive effects of brain oxidative metabolism on aggression. Proc. Natl. Acad. Sci. U.S.A. 111,

12533–12537.

Liang, Z.S., Mattila, H.R., Rodriguez-Zas, S.L., Southey, B.R., Seeley, T.D., Robinson, G.E., 2014.

Comparative brain transcriptomic analyses of scouting across distinct behavioural and ecological

contexts in honeybees. Proc. R. Soc. B 281.

Liang, Z.S., Trang, N., Mattila, H.R., Rodriguez-Zas, S.L., Seeley, T.D., Robinson, G.E., 2012.

Molecular determinants of scouting behavior in honey bees. Science 335, 1225–1228.

Libbrecht, R., Corona, M., Wende, F., Azevedo, D.O., Serrao, J.E., Keller, L., 2013. Interplay

between insulin signaling, juvenile hormone, and vitellogenin regulates maternal effects on

polyphenism in ants. Proc. Natl. Acad. Sci. U.S.A. 110, 11050–11055.

Libbrecht, R., Oxley, P.R., Keller, L., Kronauer, D.J.C., 2016. Robust DNA methylation in the clonal

raider ant brain. Curr. Biol. 26, 391–395.

Linksvayer, T.A., Kaftanoglu, O., Akyol, E., Blatch, S., Amdam, G.V., Page Jr., R.E., 2011. Larval

and nurse worker control of developmental plasticity and the evolution of honey bee queen-worker

dimorphism. J. Evol. Biol. 24, 1939–1948.

Lu, H.L., Pietrantonio, P.V., 2011. Insect insulin receptors: insights from sequence and caste

expression analyses of two cloned hymenopteran insulin receptor cDNAs from the ﬁre ant. Insect

Mol. Biol. 20, 637–649.

Lutz, C.C., Rodriguez-Zas, S.L., Fahrbach, S.E., Robinson, G.E., 2012. Transcriptional response to

foraging experience in the honey bee mushroom bodies. Dev. Neurobiol. 72, 153–166.

Lyko, F., Foret, S., Kucharski, R., Wolf, S., Falckenhayn, C., Maleszka, R., 2010. The honey bee

epigenomes: differential methylation of brain DNA in queens and workers. PLoS Biol. 8.

Mao, W., Schuler, M.A., Berenbaum, M.R., 2015. A dietary phytochemical alters caste-associated

gene expression in honey bees. Sci. Adv. 1.

Martinez, T., Wheeler, D., 1991. Effect of the queen, brood and worker caste on hemolymph

vitellogenin titer in Camponotus festinatus workers. J. Insect Physiol. 37, 347–352.

Masuoka, Y., Yaguchi, H., Suzuki, R., Maekawa, K., 2015. Knockdown of the juvenile hormone

receptor gene inhibits soldier-speciﬁc morphogenesis in the damp-wood termite Zootermopsis

nevadensis (Isoptera: Archotermopsidae). Insect Biochem. Mol. Biol. 64, 25–31.

Matsuyama, S., Nagao, T., Sasaki, K., 2015. Consumption of tyrosine in royal jelly increases brain

levels of dopamine and tyramine and promotes transition from normal to reproductive workers in

queenless honey bee colonies. General Comp. Endocrinol. 211, 1–8.

Mello, T.R.P., Aleixo, A.C., Pinheiro, D.G., Nunes, F.M.F., Bitondi, M.M.G., Hartfelder, K., Barchuk,

A.R., Simoes, Z.P., 2014. Developmental regulation of ecdysone receptor (EcR) and EcR-controlled

gene expression during pharate-adult development of honeybees (Apis mellifera). Front. Genet. 5,

445. Article No.: 445.

Menzel, R., Muller, U., 1996. Learning and memory in honeybees: from behavior to neural

substrates. Annu. Rev. Neurosci. 19, 379–404.

Mercer, A.R., Menzel, R., 1982. The effects of biogenic-amines on conditioned and unconditioned

responses to olfactory stimuli in the honeybee Apis mellifera. J. Comp. Physiol. 145, 363–368.

Michener, C.D., 1969. Comparative social behavior of bees. Annu. Rev. Entomol. 14, 299–342.

Michener, C.D., 1974. The Social Behavior of Bees : A Comparative Study. Belknap Press,

Cambridge, Mass, p. 404.

Mikheyev, A.S., Linksvayer, T.A., 2015. Genes associated with ant social behavior show distinct

transcriptional and evolutionary patterns. eLife 4.

Miura, T., Scharf, M.E., 2011. Molecular Basis Underlying Caste Differentiation in Termites, pp.

211–253.

Monaghan, E.P., Glickman, S.E., 1992. Hormones and aggressive behavior. In: Becker, J.B.,

Breedlove,

S.M., Crews, D. (Eds.), Behavioral Endocrinology. MIT Press. Monastirioti, M., Linn, C.E., White, K.,

1996. Characterization of Drosophila tyramine beta-hydroxylase gene and isolation of mutant ﬂies

lacking octopamine.

J. Neurosci. 16, 3900–3911.

Morton, G.J., Cummings, D.E., Baskin, D.G., Barsh, G.S., Schwartz, M.W., 2006. Central nervous

system control of food intake and body weight. Nature 443, 289–295.

Muller, W.J., Hepburn, H.R., 1992. Temporal and spatial patterns of wax secretion and related

behavior in the division-of-labor of the honeybee (Apis mellifera capensis). J. Comp. Physiol. 171,

111–115.

Muller, W.J., Hepburn, H.R., 1994. Juvenile-hormone-III and wax secretion in honeybees (Apis

mellifera capensis). J. Insect Physiol. 40, 873–881.

Munch, D., Ihle, K.E., Salmela, H., Amdam, G.V., 2015. Vitellogenin in the honey bee brain: atypical

localization of a reproductive protein that promotes longevity. Exp. Gerontol. 71, 103–108.

Mutti, N.S., Dolezal, A.G., Wolschin, F., Mutti, J.S., Gill, K.S., Amdam, G.V., 2011. IRS and TOR

nutrient-signaling pathways act via juvenile hormone to inﬂuence honey bee caste fate. J. Exp.

Biol. 214, 3977–3984.

Naeger, N.L., Peso, M., Even, N., Barron, A.B., Robinson, G.E., 2013. Altruistic behavior by egg-

laying worker honeybees. Curr. Biol. 23, 1574–1578.

Naeger, N.L., Van Nest, B.N., Johnson, J.N., Boyd, S.D., Southey, B.R., RodriguezZas, S.L., Moore,

D., Robinson, G.E., 2011. Neurogenomic signatures of spatiotemporal memories in time-trained

forager honey bees. J. Exp. Biol. 214, 979–987.

Nassel, D.R., Kubrak, O.I., Liu, Y., Luo, J., Lushchak, O.V., 2013. Factors that regulate insulin

producing cells and their output in Drosophila. Front. Physiol. 4.

Naug, D., Gadagkar, R., 1998. Division of labor among a cohort of young individuals in a primitively

eusocial wasp. Insectes Soc. 45.

Nelson, C.M., Ihle, K.E., Fondrk, M.K., Page Jr., R.E., Amdam, G.V., 2007. The gene vitellogenin

has multiple coordinating effects on social organization. PLoS Biol. 5, 673–677.

Nijhout, H.F., 1975. Threshold size for metamorphosis in tobacco hornworm, Manduca sexta (L.).

Biol. Bull. 149, 214–225.

Nijhout, H.F., 1994. Insectþ Hormonesþ xi 267 pp.–xi 267 pp.

Nijhout, H.F., Wheeler, D.E., 1982. Juvenile-hormone and the physiological-basis of insect

polymorphisms. Q. Rev. Biol. 57, 109–133.

Norman, V.C., Hughes, W.O.H., 2016. Behavioural effects of juvenile hormone and their inﬂuence on

division of labour in leaf-cutting ant societies. J. Exp. Biol. 219, 8–11. Nowak, M.A., Tarnita, C.E.,

Wilson, E.O., 2010. The evolution of eusociality. Nature

466, 1057–1062.

Odonnell, S., Jeanne, R.L., 1993. Methoprene accelerates age polyethism in workers of a social

wasp (Polybia occidentalis). Physiol. Entomol. 18, 189–194.

Oi, C.A., van Zweden, J.S., Oliveira, R.C., Van Oystaeyen, A., Nascimento, F.S., Wenseleers, T.,

2015. The origin and evolution of social insect queen pheromones: novel hypotheses and

outstanding problems. Bioessays 37, 808–821.

Okada, Y., Sasaki, K., Miyazaki, S., Shimoji, H., Tsuji, K., Miura, T., 2015. Social dominance and

reproductive differentiation mediated by dopaminergic signaling in a queenless ant. J. Exp. Biol.

218, 1091–1098.

Oldroyd, B.P., Beekman, M., 2008. Effects of selection for honey bee worker reproduction on

foraging traits. PLoS Biol. 6, 463–470.

Oliveira, R.C., Oi, C.A., do Nascimento, M.M.C., Vollet-Neto, A., Alves, D.A., Campos, M.C.,

Nascimento, F., Wenseleers, T., 2015. The origin and evolution of queen and fertility signals in

Corbiculate bees. BMC Evol. Biol. 15.

Oliveira, R.F., 2009. Social behavior in context: hormonal modulation of behavioral plasticity and

social competence. Integr. Comp. Biol. 49, 423–440. de Oliveira Tozetto, S., Rachinsky, A., Engels, W., 1997. Juvenile hormone promotes

ﬂight activity in drones (Apis mellifera carnica). Apidologie 28.

Ono, S., 1982. Effect of juvenile-hormone on the caste determination in the ant, Pheidole fervida

smith (Hymenoptera, Formicidae). Appl. Entomol. Zool. 17, 1–7. Oster, G.F., Wilson, E.O., 1978.

Caste and ecology in the social insects. Monogr.

Popul. Biol. 12, 1–352.

Oxley, P.R., Ji, L., Fetter-Pruneda, I., McKenzie, S.K., Li, C., Hu, H., Zhang, G., Kronauer, D.J.C.,

2014. The genome of the clonal raider ant Cerapachys biroi. Curr. Biol. 24, 451–458.

Oxley, P.R., Thompson, G.J., Oldroyd, B.P., 2008. Four quantitative trait loci that inﬂuence worker

sterility in the honeybee (Apis mellifera). Genetics 179, 1337–1343.

Page, R.E., Amdam, G.V., 2007. The making of a social insect: developmental architectures of

social design. Bioessays 29, 334–343.

Page, R.E., Robinson, G.E., Britton, D.S., Fondrk, M.K., 1992. Genotypic variability for rates of

behavioral-development in worker honeybees (Apis mellifera L.). Behav. Ecol. 3, 173–180.

Page, R.E., Rueppell, O., Amdam, G.V., 2012. Genetics of reproduction and regulation of honeybee

(Apis mellifera L.) social behavior. In: Bassler, B.L. (Ed.), Annual Review of Genetics, vol. 46.

Annual Reviews, Palo Alto, pp. 97–119.

Pamminger, T., Treanor, D., Hughes, W.O.H., 2016. Pleiotropic effects of juvenile hormone in ant

queens and the escape from the reproduction immunocompetence trade-off. Proc. R. Soc. B 283.

Pandey, A., Bloch, G., 2015. Juvenile hormone and ecdysteroids as major regulators of brain and

behavior in bees. Curr. Opin. Insect Sci. 12, 26–37.

Pankiw, T., 2004. Worker honey bee pheromone regulation of foraging ontogeny.

Naturwissenschaften 91, 178–181.

Pankiw, T., Huang, Z.Y., Winston, M.L., Robinson, G.E., 1998a. Queen mandibular gland

pheromone inﬂuences worker honey bee (Apis mellifera L.) foraging ontogeny and juvenile

hormone titers. J. Insect Physiol. 44, 685–692.

Pankiw, T., Page, R.E., 1999. The effect of genotype, age, sex, and caste on response thresholds to

sucrose and foraging behavior of honey bees (Apis mellifera L.).

J. Comp. Physiol. A 185, 207–213.

Pankiw, T., Page, R.E., 2003. Effect of pheromones, hormones, and handling on sucrose response

thresholds of honey bees (Apis mellifera L.). J. Comp. Physiol. A 189, 675–684.

Pankiw, T., Page, R.E., Fondrk, M.K., 1998b. Brood pheromone stimulates pollen foraging in honey

bees (Apis mellifera). Behav. Ecol. Sociobiol. 44, 193–198.

Passera, L., Vargo, E.L., Keller, L., 1991. Queen number in ants and its consequences for social-

organization. Annee Biol. 30, 137–173.

Patalano, S., Vlasova, A., Wyatt, C., Ewels, P., Camara, F., Ferreirab, P.G., Asher, C.L., Jurkowski,

T.P., Segonds-Pichon, A., Bachman, M., Gonzalez-Navarrete, I., Minoche, A.E., Krueger, F.,

Lowy, E., Marcet-Houben, M., Rodriguez-Ales, J.L., Nascimento, F.S., Balasubramanian, S.,

Gabaldon, T., Tarver, J.E., Andrews, S., Himmelbauer, H., Hughes, W.O.H., Guigo, R., Reik, W.,

Sumner, S., 2015. Molecular signatures of plastic phenotypes in two eusocial insect species with

simple societies. Proc. Natl. Acad. Sci. U.S.A. 112, 13970–13975.

Patel, A., Fondrk, M.K., Kaftanoglu, O., Emore, C., Hunt, G., Frederick, K., Amdam, G.V., 2007.

The making of a queen: TOR pathway is a key player in diphenic caste development. PLoS One

2, e509.

Penick, C.A., Brent, C.S., Dolezal, K., Liebig, J., 2014. Neurohormonal changes associated with

ritualized combat and the formation of a reproductive hierarchy in the ant Harpegnathos saltator.

J. Exp. Biol. 217, 1496–1503.

Penick, C.A., Liebig, J., Brent, C.S., 2011. Reproduction, dominance, and caste: endocrine proﬁles

of queens and workers of the ant Harpegnathos saltator.

J. Comp. Physiol. A 197.

Pereboom, J.J.M., 2000. The composition of larval food and the signiﬁcance of exocrine

secretions in the bumblebee Bombus terrestris. Insectes Soc. 47, 11–20. Pfaff, D.W., 1999. Drive:

Neurobiological and Molecular Mechanisms of Sexual

Motivation. MIT Press.

Phamdelegue, M.H., Trouiller, J., Caillaud, C.M., Roger, B., Masson, C., 1993. Effect of queen

pheromone on worker bees of different ages behavioral and electrophysiological responses.

Apidologie 24, 267–281.

Pinto, L.Z., Bitondi, M.M.G., Simoes, Z.L.P., 2000. Inhibition of vitellogenin synthesis in Apis

mellifera workers by a juvenile hormone analogue, pyriproxyfen. J. Insect Physiol. 46, 153–160.

Powell, S., Tschinkel, W.R., 1999. Ritualized conﬂict in Odontomachus brunneus and the generation

of interaction-based task allocation: a new organizational mechanism in ants. Anim. Behav. 58,

965–972.

Rachinsky, A., Strambi, C., Strambi, A., Hartfelder, K., 1990. Caste and metamorphosis hemolymph

titers of juvenile-hormone and ecdysteroids in last instar honeybee larvae. General Comp.

Endocrinol. 79, 31–38.

Raikhel, A.S., Brown, M.R., Belles, X., Gilbert, L.I., Iatrou, K., Gill, S.S., 2005. Hormonal control of

reproductive processes. In: Comprehensive Molecular Insect Science. Endocrinology, vol. 3, pp.

433–491.

Rajakumar, R., San Mauro, D., Dijkstra, M.B., Huang, M.H., Wheeler, D.E., HiouTim, F., Khila, A.,

Cournoyea, M., Abouheif, E., 2012. Ancestral developmental potential facilitates parallel evolution

in ants. Science 335, 79–82.

Ramamurty, P.S., Engels, W., 1977. Allatectomy and juvenile hormone effects on synthesis and

incorporation of vitellogenin in honeybee-queen (Apis mellifica). Zool. Jahrb. Allg. Zool. Physiol. Tiere

81, 165–176.

Ratnieks, F.L.W., Visscher, P.K., 1989. Worker policing in the honeybee. Nature 342, 796–797.

Reeve, H.K., 1991. Polistes. xvii 678p.. In: Ross, K.G., Matthews, R.W.þ (Eds.), The Social Biology

of Wasps. Cornell University Press, Ithaca, New York, USA; London, England, UK, pp. 99–148.

Illus. Maps.

Rehan, S.M., Toth, A.L., 2015. Climbing the social ladder: the molecular evolution of sociality.

Trends Ecol. Evol. 30, 426–433.

Ringo, J.M., Pfaff, D.W., Arnold, A.P., Etgen, A.M., Fahrbach, S.E., Rubin, R.T., 2002. Hormonal

regulation of sexual behavior in insects. Horm. Brain Behav. 3, 93–114.

Rittschof, C.C., Bukhari, S.A., Sloofman, L.G., Troy, J.M., Caetano-Anolles, D., Cash-Ahmed, A.,

Kent, M., Lu, X., Sanogo, Y.O., Weisner, P.A., Zhang, H., Bell, A.M., Ma, J., Sinha, S., Robinson,

G.E., Stubbs, L., 2014. Neuromolecular responses to social challenge: common mechanisms

across mouse, stickleback ﬁsh, and honey bee. Proc. Natl. Acad. Sci. U.S.A. 111, 17929–17934.

Rittschof, C.C., Robinson, G.E., 2014. Genomics: moving behavioural ecology beyond the

phenotypic gambit. Anim. Behav. 92, 263–270.

Rittschof, C.C., Robinson, G.E., 2016. Behavioral genetic toolkits: towards the evolutionary origin

of complex phenotypes. Curr. Top. Dev. Biol. 119, 157–204. Robinson, G.E., 1985. Effects of a

juvenile-hormone analog on honey bee foraging behavior and alarm pheromone production. J.

Insect Physiol. 31,

277–282.

Robinson, G.E., 1987a. Modulation of alarm pheromone perception in the honey-bee evidence for

division-of-labor based on hormonally regulated response thresholds.

J. Comp. Physiol. A 160, 613–619.

Robinson, G.E., 1987b. Regulation of honey-bee age polyethism by juvenile-hormone.

Behav. Ecol. Sociobiol. 20, 329–338.

Robinson, G.E., 1992. Regulation of division of labor in insect societies. Annu. Rev.

Entomol. 37, 637–665.

Robinson, G.E., 2002. Behavioral plasticity in the honey bee: ontogenetic and social regulation of

gene expression in the brain. Soc. Neurosci. Abstr. Viewer Itiner. Plan. Abstract No. 109.

Robinson, G.E., Heuser, L.M., LeConte, Y., Lenquette, F., Hollingworth, R.M., 1999.

Neurochemicals aid bee nestmate recognition. Nature 399, 534–535.

Robinson, G.E., Page, R.E., Strambi, C., Strambi, A., 1989. Hormonal and genetic-control of

behavioral integration in honey bee colonies. Science 246, 109–111.

Robinson, G.E., Page, R.E., Strambi, C., Strambi, A., 1992a. Colony integration in honey bees

mechanisms of behavioral reversion. Ethology 90, 336–348.

Robinson, G.E., Strambi, C., Strambi, A., Feldlaufer, M.F., 1991. Comparison of juvenile-hormone

and ecdysteroid hemolymph titers in adult worker and queen honey-bees (Apis mellifera). J.

Insect Physiol. 37, 929–935.

Robinson, G.E., Strambi, C., Strambi, A., Huang, Z.Y., 1992b. Reproduction in worker honey-bees

is associated with low juvenile-hormone titers and rates of biosynthesis. General Comp.

Endocrinol. 87, 471–480.

Robinson, G.E., Vargo, E.L., 1997. Juvenile hormone in adult eusocial hymenoptera: gonadotropin

and behavioral pacemaker. Archiv. Insect Biochem. Physiol. 35, 559–583.

Roeder, T., 1999. Octopamine in invertebrates. Prog. Neurobiol. 59, 533–561. Romiguier, J.,

Cameron, S.A., Woodard, S.H., Fischman, B.J., Keller, L., Praz, C.J., 2016. Phylogenomics

controlling for base compositional bias reveals a single origin of eusociality in corbiculate bees.

Mol. Biol. Evol. 33, 670–678.

Ronai, I., Barton, D.A., Oldroyd, B.P., Vergoz, V., 2015. Regulation of oogenesis in honey bee

workers via programed cell death. J. Insect Physiol. 81, 36–41.

Roseler, P.F., 1991. Reproductive competition during colony establishment. In: The Social Biology

of Wasps, pp. 309–335.

Röseler, P.F., 1977. Juvenile hormone control of oogenesis in bumblebee workers,

Bombus terrestris. J. Insect Physiol. 23, 985–992.

Röseler, P.F., 1985. Endocrine basis of dominance and reproduction in polistine paper wasps.

Fortschr. Zool. 31, 259–272.

Röseler, P.F., Röseler, I., 1978. Studies on regulation of juvenile hormone titer in bumblebee

workers, Bombus terrestris. J. Insect Physiol. 24, 707–713.

Röseler, P.F., Röseler, I., 1988. Inﬂuence of juvenile-hormone on fat-body metabolism in

ovariolectomized queens of the bumblebee, Bombus terrestris. Insect Biochem. 18, 557–563.

Röseler, P.F., Röseler, I., Strambi, A., 1985. Role of ovaries and ecdysteroids in dominance

hierarchy establishment among foundresses of the primitively social wasp, Polistes gallicus.

Behav. Ecol. Sociobiol. 18, 9–13.

Röseler, P.F., Röseler, I., Strambi, A., 1986. Studies of the dominance hierarchy in the paper wasp,

Polistes gallicus (L.) (Hymenoptera-Vespidae). Monit. Zool. Ital. Ital. J. Zool. 20, 283–290.

Röseler, P.F., Röseler, I., Strambi, A., Augier, R., 1984. Inﬂuence of insect hormones on the

establishment of dominance hierarchies among foundresses of the paper wasp, Polistes gallicus.

Behav. Ecol. Sociobiol. 15, 133–142.

Ross, K.G., Keller, L., 1995. Ecology and evolution of social-organization insights from ﬁre ants and

other highly eusocial insects. Annu. Rev. Ecol. Syst. 26, 631–656.

Ruffner, M.E., Cromarty, S.I., Cooper, R.L., 1999. Depression of synaptic efﬁcacy in highand low-

output Drosophila neuromuscular junctions by the molting hormone (20-HE). J. Neurophysiol. 81,

788–794.

Rutz, W., Gerig, L., Wille, H., Luscher, M., 1976. Function of juvenile hormone in adult worker

honeybees, Apis mellifera. J. Insect Physiol. 22, 1485–1491.

Sadd, B.M., Barribeau, S.M., Bloch, G., de Graaf, D.C., Dearden, P., Elsik, C.G., Gadau, J.,

Grimmelikhuijzen, C.J.P., Hasselmann, M., Lozier, J.D., Robertson, H.M., Smagghe, G., Stolle, E.,

Van Vaerenbergh, M., Waterhouse, R.M., BornbergBauer, E., Klasberg, S., Bennett, A.K.,

Camara, F., Guigo, R., Hoff, K., Mariotti, M., Munoz-Torres, M., Murphy, T., Santesmasses, D.,

Amdam, G.V., Beckers, M., Beye, M., Biewer, M., Bitondi, M.M.G., Blaxter, M.L., Bourke, A.F.G.,

Brown, M.J.F., Buechel, S.D., Cameron, R., Cappelle, K., Carolan, J.C., Christiaens, O.,

Ciborowski, K.L., Clarke, D.F., Colgan, T.J., Collins, D.H., Cridge, A.G., Dalmay, T., Dreier, S., du

Plessis, L., Duncan, E., Erler, S., Evans, J., Falcon, T., Flores, K., Freitas, F.C., PFuchikawa, T.,

Gempe, T., Hartfelder, K., Hauser, F., Helbing, S., Humann, F.C., Irvine, F., Jermiin, L.S.,

Johnson, C.E., Johnson, R.M., Jones, A.K., Kadowaki, T., Kidner, J.H., Koch, V., Koehler, A.,

Kraus, F.B., Lattorff, H.M.G., Leask, M., Lockett, G.A., Mallon, E.B., Antonio, D.S.M., Marxer, M.,

Meeus, I., Moritz, R.F.A., Nair, A., Napﬂin, K., Nissen, I., Niu, J., Nunes, F.M.F., Oakeshott, J.G.,

Osborne, A., Otte, M., Pinheiro, D.G., Rossie, N., Rueppell, O., Santos, C.G., Schmid-Hempel, R.,

Schmitt, B.D., Schulte, C., Simoes, Z.L.P., Soares, M.P.M., Swevers, L., Winnebeck, E.C., Wolschin,

F., Yu, N., Zdobnov, E.M., Aqrawi, P.K., Blankenburg, K.P., Coyle, M., Francisco, L., Hernandez,

A.G., Holder, M., Hudson, M.E., Jackson, L., Jayaseelan, J., Joshi, V., Kovar, C., Lee, S.L., Mata,

R., Mathew, T., Newsham, I.F., Ngo, R., Okwuonu, G., Pham, C., Pu, L.L., Saada, N., Santibanez,

J., Simmons, D., Thornton, R., Venkat, A., Walden, K.K.O., Wu, Y.-Q., Debyser, G., Devreese, B.,

Asher, C., Blommaert, J., Chipman, A.D., Chittka, L., Fouks, B., Liu, J., O’Neill, M.P., Sumner, S.,

Puiu, D., Qu, J., Salzberg, S.L., Scherer, S.E., Muzny, D.M., Richards, S., Robinson, G.E., Gibbs,

R.A., Schmid-Hempel, P., Worley, K.C., 2015. The genomes of two key bumblebee species with

primitive eusocial organization. Genome Biol. 16.

Sakagami, S.F., Maeta, Y., 1987. Sociality, induced and/or natural, in the basically solitary small

carpenter bees (Ceratina). Anim. Soc. Theor. Facts 1–16.

Sandoz, J.-C., 2013. Neural correlates of olfactory learning in the primary olfactory center of the

honeybee brain the antennal lobe. In: Menzel, R., Benjamin, P.R. (Eds.), Invertebrate Learning

and Memory, vol. 22, pp. 416–432.

Sasaki, K., Harano, K.I., 2007. Potential effects of tyramine on the transition to reproductive workers

in honeybees (Apis mellifera L.). Physiol. Entomol. 32, 194–198.

Sasaki, K., Matsuyama, S., Harano, K., Nagao, T., 2012. Caste differences in dopamine-related

substances and dopamine supply in the brains of honeybees (Apis mellifera L.). General Comp.

Endocrinol. 178, 46–53.

Sasaki, K., Nagao, T., 2002. Brain tyramine and reproductive states of workers in honeybees. J.

Insect Physiol. 48, 1075–1085.

Sasaki, K., Yamasaki, K., Nagao, T., 2007. Neuro-endocrine correlates of ovarian development and

egg-laying behaviors in the primitively eusocial wasp (Polistes chinensis). J. Insect Physiol. 53,

940–949.

Schal, C., Holbrook, G.L., Bachmann, J.A.S., Sevala, V.L., 1997. Reproductive biology of the German

cockroach, Blattella germanica: juvenile hormone as a pleiotropic master regulator. Archiv. Insect

Biochem. Physiol. 35, 405–426.

Scheiner, R., Barnert, M., Erber, J., 2003. Variation in water and sucrose responsiveness during the

foraging season affects proboscis extension learning in honey bees. Apidologie 34, 67–72.

Scheiner, R., Kulikovskaja, L., Thamm, M., 2014a. The honey bee tyramine receptor AmTYR1 and

division of foraging labour. J. Exp. Biol. 217, 1215–1217.

Scheiner, R., Toteva, A., Reim, T., Sovik, E., Barron, A.B., 2014b. Differences in the phototaxis of

pollen and nectar foraging honey bees are related to their octopamine brain titers. Front. Physiol.

Schulz, D.J., Huang, Z.Y., Robinson, G.E., 1998. Effects of colony food shortage on behavioral

development in honey bees. Behav. Ecol. Sociobiol. 42, 295–303.

Schulz, D.J., Robinson, G.E., 1999. Biogenic amines and division of labor in honey bee colonies:

behaviorally related changes in the antennal lobes and age-related changes in the mushroom

bodies. J. Comp. Physiol. A 184, 481–488.

Schulz, D.J., Robinson, G.E., 2001. Octopamine inﬂuences division of labor in honey bee colonies.

J. Comp. Physiol. A 187, 53–61.

Schulz, D.J., Sullivan, J.P., Robinson, G.E., 2002. Juvenile hormone and octopamine in the

regulation of division of labor in honey bee colonies. Horm. Behav. 42, 222–231.

Seehuus, S.-C., Norberg, K., Krekling, T., Fondrk, K., Amdam, G.V., 2007. Immunogold localization

of vitellogenin in the ovaries, hypopharyngeal glands and head fat bodies of honeybee workers,

Apis mellifera. J. Insect Sci. 7.

Shingleton, A.W., Frankino, W.A., Flatt, T., Nijhout, H.F., Emlen, D.J., 2007. Size and shape: the

developmental regulation of static allometry in insects. Bioessays 29, 536–548.

Shorter, J.R., Tibbetts, E.A., 2009. The effect of juvenile hormone on temporal polyethism in the

paper wasp Polistes dominulus. Insectes Soc. 56, 7–13.

Shpigler, H., Amsalem, E., Huang, Z.Y., Cohen, M., Siegel, A.J., Hefetz, A., Bloch, G., 2014.

Gonadotropic and physiological functions of juvenile hormone in bumblebee (Bombus terrestris)

workers. PLoS One 9.

Shpigler, H., Tamarkin, M., Gruber, Y., Poleg, M., Siegel, A.J., Bloch, G., 2013. Social inﬂuences on

body size and developmental time in the bumblebee Bombus terrestris. Behav. Ecol. Sociobiol.

67, 1601–1612.

Simola, D.F., Graham, R.J., Brady, C.M., Enzmann, B.L., Desplan, C., Ray, A., Zwiebel, L.J.,

Bonasio, R., Reinberg, D., Liebig, J., Berger, S.L., 2016. Epigenetic (re)programming of caste-

speciﬁc behavior in the ant Camponotus floridanus. Science (New York, NY) 351.

Simola, D.F., Wissler, L., Donahue, G., Waterhouse, R.M., Helmkampf, M., Roux, J., Nygaard, S.,

Glastad, K.M., Hagen, D.E., Viljakainen, L., Reese, J.T., Hunt, B.G., Graur, D., Elhaik, E.,

Kriventseva, E.V., Wen, J., Parker, B.J., Cash, E., Privman, E., Childers, C.P., Munoz-Torres, M.C.,

Boomsma, J.J., Bornberg-Bauer, E., Currie, C.R., Elsik, C.G., Suen, G., Goodisman, M.A.D.,

Keller, L., Liebig, J., Rawls, A., Reinberg, D., Smith, C.D., Smith, C.R., Tsutsui, N., Wurm, Y.,

Zdobnov, E.M., Berger, S.L., Gadau, J., 2013a. Social insect genomes exhibit dramatic evolution

in gene composition and regulation while preserving regulatory features linked to sociality.

Genome Res. 23, 1235–1247.

Simola, D.F., Ye, C., Mutti, N.S., Dolezal, K., Bonasio, R., Liebig, J., Reinberg, D., Berger, S.L.,

2013b. A chromatin link to caste identity in the carpenter ant Camponotus floridanus. Genome

Res. 23, 486–496.

Sinha, S., Ling, X., Whitﬁeld, C.W., Zhai, C., Robinson, G.E., 2006. Genome scan for cis-regulatory

DNA motifs associated with social behavior in honey bees. Proc. Natl. Acad. Sci. U.S.A. 103,

16352–16357.

Slessor, K.N., Kaminski, L.A., Winston, M.L., 1998. The essence of royalty queen honey bee

pheromones. In: Abstracts of Papers of the American Chemical Society, vol. 195, 55–AGRO.

Smith, A.R., Kapheim, K.M., Perez-Ortega, B., Brent, C.S., Wcislo, W.T., 2013. Juvenile hormone

levels reﬂect social opportunities in the facultatively eusocial sweat bee Megalopta genalis

(Hymenoptera: Halictidae). Horm. Behav. 63, 1–4.

Smith, C.R., Toth, A.L., Suarez, A.V., Robinson, G.E., 2008. Genetic and genomic analyses of the

division of labour in insect societies. Nat. Rev. Genet. 9, 735–748. Sommer, K., Holldobler, B.,

1995. Colony founding by queen association and determinants of reduction in queen number in

the ant Lasius niger. Anim. Behav. 50, 287–294.

Spannhoff, A., Kim, Y.K., Raynal, N.J.M., Gharibyan, V., Su, M.-B., Zhou, Y.-Y., Li, J., Castellano,

S., Sbardella, G., Issa, J.-P.J., Bedford, M.T., 2011. Histone deacetylase inhibitor activity in royal

jelly might facilitate caste switching in bees. EMBO Rep. 12, 238–243.

Standage, D.S., Berens, A.J., Glastad, K.M., Severin, A.J., Brendel, V.P., Toth, A.L., 2016. Genome,

transcriptome and methylome sequencing of a primitively eusocial wasp reveal a greatly reduced

DNA methylation system in a social insect. Mol. Ecol. 25 (8), 1769–1784.

Stout, J.F., Atkins, G.J., Bronsert, M., Hao, J., Walikonis, R., Pfaff, D.W., Arnold, A.P., Etgen, A.M.,

Fahrbach, S.E., Rubin, R.T., 2002. Inﬂuence of juvenile hormone III on the development and

plasticity of the responsiveness of female crickets to calling males through control of the response

properties of identiﬁed auditory neurons. Horm. Brain Behav. 3, 167–193.

Strambi, A., Strambi, C., Cayre, M., 1997. Hormonal control of reproduction and reproductive

behavior in crickets. Archiv. Insect Biochem. Physiol. 35, 393–404. Strambi, A., Strambi, C., de

Reggi, M., 1977. Ecdysones and ovarian physiology in the adult wasp Polistes gallicus. Proc.

Congr. Int. U.N. Study Soc. Ins. 8, 19–20 (illust.).

Sullivan, J.P., Jassim, O., Fahrbach, S.E., Robinson, G.E., 2000. Juvenile hormone paces

behavioral development in the adult worker honey bee. Horm. Behav. 37, 1–14.

Suryanarayanan, S., Hermanson, J.C., Jeanne, R.L., 2011. A mechanical signal biases caste

development in a social wasp. Curr. Biol. 21, 231–235.

Suzzoni, J.P., Passera, L., Strambi, A., 1980. Ecdysteroid titer and caste determination in the ant,

Pheidole pallidula (NYL) (Hymenoptera, Formicidae). Experientia 36, 1228–1229.

Taylor, D.J., Robinson, G.E., Logan, B.J., Laverty, R., Mercer, A.R., 1992. Changes in brain amine

levels associated with the morphological and behavioral-development of the worker honeybee. J.

Comp. Physiol. A 170, 715–721.

Terrapon, N., Li, C., Robertson, H.M., Ji, L., Meng, X., Booth, W., Chen, Z., Childers, C.P., Glastad,

K.M., Gokhale, K., Gowin, J., Gronenberg, W., Hermansen, R.A., Hu, H., Hunt, B.G., Huylmans,

A.K., Khalil, S.M.S., Mitchell, R.D., Munoz-Torres, M.C., Mustard, J.A., Pan, H., Reese, J.T.,

Scharf, M.E., Sun, F., Vogel, H., Xiao, J., Yang, W., Yang, Z., Yang, Z., Zhou, J., Zhu, J., Brent,

C.S., Elsik, C.G., Goodisman, M.A.D., Liberles, D.A., Roe, R.M., Vargo, E.L., Vilcinskas, A.,

Wang, J., Bornberg-Bauer, E., Korb, J., Zhang, G., Liebig, J., 2014. Molecular traces of alternative

social organization in a termite genome. Nat. Commun. 5.

Thamm, M., Balfanz, S., Scheiner, R., Baumann, A., Blenau, W., 2010. Characterization of the 5-

HT1A receptor of the honeybee (Apis mellifera) and involvement of serotonin in phototactic

behavior. Cell. Mol. Life Sci. 67, 2467–2479.

Thompson, G.J., Kucharski, R., Maleszka, R., Oldroyd, B.P., 2006. Towards a molecular deﬁnition

of worker sterility: differential gene expression and reproductive plasticity in honey bees. Insect

Mol. Biol. 15, 637–644.

Thompson, G.J., Kucharski, R., Maleszka, R., Oldroyd, B.P., 2008. Genome-wide analysis of genes

related to ovary activation in worker honey bees. Insect Mol. Biol. 17, 657–665.

Tibbetts, E.A., Huang, Z.Y., 2010. The challenge hypothesis in an insect: juvenile hormone

increases during reproductive conﬂict following queen loss in polistes wasps. Am. Nat. 176, 123–

130.

Tibbetts, E.A., Izzo, A.S., 2009. Endocrine mediated phenotypic plasticity: conditiondependent

effects of juvenile hormone on dominance and fertility of wasp queens. Horm. Behav. 56, 527–

531.

Tibbetts, E.A., Levy, S., Donajkowski, K., 2011. Reproductive plasticity in Polistes paper wasp

workers and the evolutionary origins of sociality. J. Insect Physiol. 57.

Tibbetts, E.A., Sheehan, M.J., 2012. The effect of juvenile hormone on Polistes wasp fertility varies

with cooperative behavior. Horm. Behav. 61, 559–564.

Tillman, J.A., Seybold, S.J., Jurenka, R.A., Blomquist, G.J., 1999. Insect pheromones

an overview of biosynthesis and endocrine regulation. Insect Biochem. Mol. Biol. 29, 481–514.

Tobe, S.S., Stay, B., 1985. Structure and regulation of the corpus allatum. Adv. Insect Physiol. 18,

305–432.

Toth, A.L., Kantarovich, S., Meisel, A.F., Robinson, G.E., 2005. Nutritional status inﬂuences socially

regulated foraging ontogeny in honey bees. J. Exp. Biol. 208, 4641–4649.

Toth, A.L., Robinson, G.E., 2005. Worker nutrition and division of labour in honeybees.

Anim. Behav. 69, 427–435.

Toth, A.L., Robinson, G.E., 2007. Evo-devo and the evolution of social behavior. Trends Genet. 23,

334–341.

Toth, A.L., Varala, K., Newman, T.C., Miguez, F.E., Hutchison, S.K., Willoughby, D.A., Simons, J.F.,

Egholm, M., Hunt, J.H., Hudson, M.E., Robinson, G.E., 2007. Wasp gene expression supports an

evolutionary link between maternal behavior and eusociality. Science 318, 441–444.

Townsend-Mehler, J.M., Dyer, F.C., Maida, K., 2010. Deciding When to Explore and When to

Persist: A Comparison of Honeybees and Bumblebees in Their Response to Downshifts in

Reward, vol. 65, pp. 305–312.

Trumbo, S.T., Pfaff, D.W., Arnold, A.P., Etgen, A.M., Fahrbach, S.E., Rubin, R.T., 2002. Hormonal

regulation of parental care in insects. Horm. Brain Behav. 3, 115–139.

Tschinkel, W.R., 1995. Stimulation of ﬁre ant queen fecundity by a highly speciﬁc brood stage. Ann.

Entomol. Soc. Am. 88, 876–882.

Ueno, T., Takeuchi, H., Kawasaki, K., Kubo, T., 2015. Changes in the gene expression proﬁles of the

hypopharyngeal gland of worker honeybees in association with worker behavior and hormonal

factors. PLoS One 10.

Van Oystaeyen, A., Oliveira, R.C., Holman, L., van Zweden, J.S., Romero, C., Oi, C.A., d’Ettorre, P.,

Khalesi, M., Billen, J., Waeckers, F., Millar, J.G., Wenseleers, T., 2014. Conserved class of queen

pheromones stops social insect workers from reproducing. Science 343, 287–290.

Vargo, E.L., 1992. Mutual pheromonal inhibition among queens in polygyne colonies of the ﬁre ant

Solenopsis invicta. Behav. Ecol. Sociobiol. 31, 205–210.

Vargo, E.L., Fletcher, D.J.C., 1989. On the relationship between queen number and fecundity in

polygyne colonies of the ﬁre ant Solenopsis invicta. Physiol. Entomol. 14, 223–232.

Vargo, E.L., Hulsey, C.D., 2000. Multiple glandular origins of queen pheromones in the

ﬁre ant Solenopsis invicta. J. Insect Physiol. 46, 1151–1159.

Vargo, E.L., Laurel, M., 1994. Studies on the mode of action of a queen primer pheromone of the

ﬁre ant Solenopsis invicta. J. Insect Physiol. 40, 601–610.

Velarde, R.A., Robinson, G.E., Fahrbach, S.E., 2009. Coordinated responses to developmental

hormones in the Kenyon cells of the adult worker honey bee brain (Apis mellifera L.). J. Insect

Physiol. 55, 59–69.

Vergoz, V., Lim, J., Oldroyd, B.P., 2012. Biogenic amine receptor gene expression in the ovarian

tissue of the honey bee Apis mellifera. Insect Mol. Biol. 21, 21–29.

Visscher, P.K., 1989. A quantitative study of worker reproduction in honey bee colonies. Behav.

Ecol. Sociobiol. 25, 247–254.

Visscher, P.K., Dukas, R., 1995. Honey-bees recognize development of nestmates ovaries. Anim.

Behav. 49, 542–544.

Von Frisch, K., 1967. The Dance Language and Orientation of Bees xiv 566p. þ

Illus. Maps.

Wagener-Hulme, C., Kuehn, J.C., Schulz, D.J., Robinson, G.E., 1999. Biogenic amines and division

of labor in honey bee colonies. J. Comp. Physiol. A 184, 471–479.

Wang, Y., Amdam, G.V., Rueppell, O., Wallrichs, M.A., Fondrk, M.K., Kaftanoglu, O., Page Jr., R.E.,

2009. PDK1 and HR46 gene homologs tie social behavior to ovary signals. PLoS One 4.

Wang, Y., Azevedo, S.V., Hartfelder, K., Amdam, G.V., 2013a. Insulin-like peptides (AmILP1 and

AmILP2) differentially affect female caste development in the honey bee (Apis mellifera L.). J.

Exp. Biol. 216, 4347–4357.

Wang, Y., Baker, N., Amdam, G.V., 2013b. RNAi-mediated double gene knockdown and gustatory

perception measurement in honey bees (Apis mellifera). J. Vis. Exp.

http://dx.doi.org/10.3791/50446.

Wang, Y., Brent, C.S., Fennern, E., Amdam, G.V., 2012a. Gustatory perception and fat body energy

metabolism are jointly affected by vitellogenin and juvenile hormone in honey bees. PLoS Genet.

8, 13.

Wang, Y., Jorda, M., Jones, P.L., Maleszka, R., Ling, X., Robertson, H.M., Mizzen, C.A., Peinado,

M.A., Robinson, G.E., 2006. Functional CpG methylation system in a social insect. Science 314,

645–647.

Wang, Y., Kocher, S.D., Linksvayer, T.A., Grozinger, C.M., Page Jr., R.E., Amdam, G.V., 2012b.

Regulation of behaviorally associated gene networks in worker honey bee ovaries. J. Exp. Biol.

215, 124–134.

Watanabe, D., Gtoh, H., Miura, T., Maekawa, K., 2014. Social interactions affecting caste

development through physiological actions in termites. Front. Physiol. 5.

Weaver, N., 1957. Effects of larval age on dimorphic differentiation of the female honey bee. Ann.

Entomol. Soc. Am. 50, 283–294.

Wehling, M., 1994. Nongenomic actions of steroid hormones. Trends Endocrinol.

Metab. 5, 347–353.

Weiner, S.A., Galbraith, D.A., Adams, D.C., Valenzuela, N., Noll, F.B., Grozinger, C.M., Toth, A.L.,

2013. A survey of DNA methylation across social insect species, life stages, and castes reveals

abundant and caste-associated methylation in a primitively social wasp. Naturwissenschaften

100, 795–799.

Weinstock, G.M., Robinson, G.E., Gibbs, R.A., Worley, K.C., Evans, J.D., Maleszka, R., Robertson,

H.M., Weaver, D.B., Beye, M., Bork, P., Elsik, C.G., Hartfelder, K., Hunt, G.J., Zdobnov, E.M.,

Amdam, G.V., Bitondi, M.M.G., Collins, A.M., Cristino, A.S., Lattorff, H.M.G., Lobo, C.H., Moritz,

R.F.A., Nunes, F.M.F., Page, R.E., Simoes, Z.L.P., Wheeler, D., Carninci, P., Fukuda, S.,

Hayashizaki, Y., Kai, C., Kawai, J., Sakazume, N., Sasaki, D., Tagami, M., Albert, S., Baggerman,

G., Beggs, K.T., Bloch, G., Cazzamali, G., Cohen, M., Drapeau, M.D., Eisenhardt, D., Emore, C.,

Ewing, M.A., Fahrbach, S.E., Foret, S., Grimmelikhuijzen, C.J.P., Hauser, F., Hummon, A.B.,

Huybrechts, J., Jones, A.K., Kadowaki, T., Kaplan, N., Kucharski, R., Leboulle, G., Linial, M.,

Littleton, J.T., Mercer, A.R., Richmond, T.A., Rodriguez-Zas, S.L., Rubin, E.B., Sattelle, D.B.,

Schlipalius, D., Schoofs, L., Shemesh, Y., Sweedler, J.V., Velarde, R., Verleyen, P., Vierstraete, E.,

Williamson, M.R., Ament, S.A., Brown, S.J., Corona, M., Dearden, P.K., Dunn, W.A., Elekonich,

M.M., Fujiyuki, T., Gattermeier, I., Gempe, T., Hasselmann, M., Kage, E., Kamikouchi, A., Kubo,

T., Kunieda, T., Lorenzen, M.D., Milshina, N.V., Morioka, M., Ohashi, K., Overbeek, R., Ross, C.A.,

Schioett, M., Shippy, T., Takeuchi, H., Toth, A.L., Willis, J.H., Wilson, M.J., Gordon, K.H.J.,

Letunic, I., Hackett, K., Peterson, J., Felsenfeld, A., Guyer, M., Solignac, M., Agarwala, R.,

Cornuet, J.M., Monnerot, M., Mougel, F., Reese, J.T., Vautrin, D., Gillespie, J.J., Cannone, J.J.,

Gutell, R.R., Johnston, J.S., Eisen, M.B., Iyer, V.N., Iyer, V., Kosarev, P., Mackey, A.J., Solovyev,

V., Souvorov, A., Aronstein, K.A., Bilikova, K., Chen, Y.P., Clark, A.G., Decanini, L.I., Gelbart,

W.M., Hetru, C., Hultmark, D., Imler, J.L., Jiang, H.B., Kanost, M., Kimura, K., Lazzaro, B.P.,

Lopez, D.L., Simuth, J., Thompson, G.J., Zou, Z., De Jong, P., Sodergren, E., Csuros, M.,

Milosavljevic, A., Osoegawa, K., Richards, S., Shu, C.L., Duret, L., Elhaik, E., Graur, D., Anzola,

J.M., Campbell, K.S., Childs, K.L., Collinge, D., Crosby, M.A., Dickens, C.M., Grametes, L.S.,

Grozinger, C.M., Jones, P.L., Jorda, M., Ling, X., Matthews, B.B., Miller, J., Mizzen, C., Peinado,

M.A., Reid, J.G., Russo, S.M., Schroeder, A.J., St Pierre, S.E., Wang, Y., Zhou, P.L., Jiang, H.Y.,

Kitts, P., Ruef, B., Venkatraman, A., Zhang, L., AquinoPerez, G., Whitﬁeld, C.W., Behura, S.K.,

Berlocher, S.H., Sheppard, W.S., Smith, D.R., Suarez, A.V., Tsutsui, N.D., Wei, X.H., Havlak, P.,

Li, B.S., Liu, Y., Jolivet, A., Lee, S., Nazareth, L.V., Pu, L.L., Thorn, R., Stolc, V., Newman, T.,

Samanta, M., Tongprasit, W.A., Claudianos, C., Berenbaum, M.R., Biswas, S., de Graaf, D.C.,

Feyereisen, R., Johnson, R.M., Oakeshott, J.G., Ranson, H., Schuler, M.A., Muzny, D., Chacko,

J., Davis, C., Dinh, H., Gill, R., Hernandez, J., Hines, S., Hume, J., Jackson, L., Kovar, C., Lewis,

L., Miner, G., Morgan, M., Nguyen, N., Okwuonu, G., Paul, H., Santibanez, J., Savery, G., Svatek,

A., Villasana, D., Wright, R., Honeybee Genome Sequencing, C, 2006. Insights into social insects

from the genome of the honeybee Apis mellifera. Nature 443, 931–949. West-Eberhard, M.J.,

1978. Polygyny and the evolution of social behavior in wasps. J. Kans. Entomol. Soc. 51, 832–

856.

West-Eberhard, M.J., Turillazzi, S., 1996. Wasp societies as microcosms for the study of

development and evolution. In: Natural History and Evolution of Paper-Wasps, pp. 290–317.

West-Eberhard, M.J., 1975. The evolution of social behavior by kin selection. Q. Rev.

Biol. 50, 1–33.

Wheeler, D.E., 1986. Developmental and physiological determinants of caste in social Hymenoptera

evolutionary implications. Am. Nat. 128, 13–34.

Wheeler, D.E., 1991. The developmental basis of worker caste polymorphism in ants.

Am. Nat. 138, 1218–1238.

100

Wheeler, D.E., Buck, N., Evans, J.D., 2006. Expression of insulin pathway genes during the period

of caste determination in the honey bee, Apis mellifera. Insect Mol. Biol. 15, 597–602.

Wheeler, D.E., Nijhout, H.F., 1981. Soldier determination in ants new role for juvenile hormone.

Science 213, 361–363.

Wheeler, D.E., Nijhout, H.F., 1983. Soldier determination in Pheidole bicarinata – effect of

methoprene on caste and size within castes. J. Insect Physiol. 29, 847–854.

Wheeler, D.E., Nijhout, H.F., 1984. Soldier determination in Pheidole bicarinata –

inhibition by adult soldiers. J. Insect Physiol. 30, 127–135.

Wheeler, M.M., Ament, S.A., Rodriguez-Zas, S.L., Robinson, G.E., 2013. Brain gene expression

changes elicited by peripheral vitellogenin knockdown in the honey bee. Insect Mol. Biol. 22, 562–

573.

Wheeler, M.M., Ament, S.A., Rodriguez-Zas, S.L., Southey, B., Robinson, G.E., 2015. Diet and

endocrine effects on behavioral maturation-related gene expression in the pars intercerebralis of

the honey bee brain. J. Exp. Biol. 218, 4005–4014.

Wheeler, M.M., Robinson, G.E., 2014. Diet-dependent gene expression in honey bees: honey vs.

sucrose or high fructose corn syrup. Sci. Rep. 4.

Whitﬁeld, C.W., Ben-Shahar, Y., Brillet, C., Leoncini, I., Crauser, D., Le Conte, Y., Rodriguez-Zas,

S., Robinson, G.E., 2006. Genomic dissection of behavioral maturation in the honey bee. Proc.

Natl. Acad. Sci. U.S.A. 103, 16068–16075.

Whitﬁeld, C.W., Cziko, A.M., Robinson, G.E., 2003. Gene expression proﬁles in the brain predict

behavior in individual honey bees. Science 302, 296–299.

Williams, D.W., Truman, J.W., 2005. Remodeling dendrites during insect metamorphosis. J.

Neurobiol. 64, 24–33.

Wilson, E.O., 1971. The Insect Societies. Belknap Press of Harvard University Press, Cambridge,

Mass, p. 548.

101

Wilson, E.O., 1975. Sociobiology : The New Synthesis. Belknap Press of Harvard University Press,

Cambridge, Mass ix, 697p.

Wilson, E.O., 1998. Consilience : The Unity of Knowledge. Distributed by Random House, ﬁrst ed.

Knopf, New York, p. 332.

Wilson, E.O., Holldobler, B., 2005. Eusociality: origin and consequences. Proc. Natl.

Acad. Sci. U.S.A. 102, 13367–13371.

Wilson, E.O., Southwood, R., 1990. Success and Dominance in Ecosystems: The Case of the Social

Insects. Ecology Institute.

Winston, M.L., 1987. The Biology of the Honey Bee. Harvard University Press, USA. Wirtz, P., 1973.

Differentiation in the honeybee larva: a histological, electronmicroscopical and physiological study

of caste induction in Apis mellifera mellifera L. Meded. Landbouwhogesch. Wagening. 73, 155.

Wojciechowski, M., Rafalski, D., Kucharski, R., Misztal, K., Maleszka, J., Bochtler, M., Maleszka, R.,

2014. Insights into DNA hydroxymethylation in the honeybee from indepth analyses of TET

dioxygenase. Open Biol. 4.

Woodard, S.H., Fischman, B.J., Venkat, A., Hudson, M.E., Varala, K., Cameron, S.A., Clark, A.G.,

Robinson, G.E., 2011. Genes involved in convergent evolution of eusociality in bees. Proc. Natl.

Acad. Sci. U.S.A. 108, 7472–7477.

Wu, Q., Wen, T., Lee, G., Park, J.H., Cai, H.N., Shen, P., 2003. Developmental control of foraging

and social behavior by the Drosophila neuropeptide Y-like system. Neuron 39, 147–161.

Wurm, Y., Wang, J., Riba-Grognuz, O., Corona, M., Nygaard, S., Hunt, B.G., Ingram, K.K., Falquet,

L., Nipitwattanaphon, M., Gotzek, D., Dijkstra, M.B., Oettler, J., Comtesse, F., Shih, C.-J., Wu,

W.-J., Yang, C.-C., Thomas, J., Beaudoing, E., Pradervand, S., Flegel, V., Cook, E.D., Fabbretti,

R., Stockinger, H., Long, L., Farmerie, W.G., Oakey, J., Boomsma, J.J., Pamilo, P., Yi, S.V.,

Heinze, J., Goodisman, M.A.D., Farinelli, L., Harshman, K., Hulo, N., Cerutti, L., Xenarios, I.,

Shoemaker, D., Keller, L., 2011. The genome of the ﬁre ant Solenopsis invicta. Proc. Natl. Acad.

Sci. U.S.A. 108, 5679–5684.

102

Wyatt, G.R., Davey, K.G., 1996. Cellular and molecular actions of juvenile hormone. 2. Roles of

juvenile hormone in adult insects. In: Evans, P.D. (Ed.), Advances in Insect Physiology, vol. 26, pp.

1–155.

Zayed, A., Robinson, G.E., 2012. Understanding the relationship between brain gene expression

and social behavior: lessons from the honey bee. Annu. Rev. Genet. 46, 591–615.

Zera, A.J., Harshman, L.G., Williams, T.D., 2007. Evolutionary endocrinology: the developing

synthesis between endocrinology and evolutionary genetics. Annu. Rev. Ecol. Evol. Syst. 38, 793–

817.

Zhu, J., Chen, L., Raikhel, A.S., 2007. Distinct roles of broad isoforms in regulation of the 20-

hydroxyecdysone effector gene, vitellogenin, in the mosquito Aedes aegypti. Mol. Cell. Endocrinol.

267, 97–105.

103

Chapter 2: Insights into the Transcriptional Architecture of Behavioral Plasticity in the Honey Bee Apis mellifera*

Abstract Honey bee colonies exhibit an age-related division of labor, with worker bees performing discrete sets of behaviors throughout their lifespan. These behavioral states are associated with distinct brain transcriptomic states, yet little is known about the regulatory mechanisms governing them. We used CAGEscan (a variant of the Cap Analysis of Gene Expression technique) for the first time to characterize the promoter regions of differentially expressed brain genes during two behavioral states (brood care (aka “nursing”) and foraging) and identified transcription factors (TFs) that may govern their expression. More than half of the differentially expressed TFs were associated with motifs enriched in the promoter regions of differentially expressed genes (DEGs), suggesting they are regulators of behavioral state. Strikingly, five TFs (NF-κB, egr, pax6, hairy, and clockwork orange) were predicted to co-regulate nearly half of the genes that were upregulated in foragers. Finally, differences in alternative TSS usage between nurses and foragers were detected upstream of 646 genes, whose functional analysis revealed enrichment for Gene Ontology terms associated with neural function and plasticity. This demonstrates for the first time that alternative TSSs are associated with stable differences in behavior, suggesting they may play a role in organizing behavioral state.

*This work was originally published as (Khamis & Hamilton et al., 2015). It has been edited from its original format to adjust section headings and figure/table numbering.

104

2.1 INTRODUCTION Due to its extensive behavioral repertoire and highly social lifestyle, the European honey bee (Apis mellifera) has been utilized as an ethological model for decades. More recently, the publication of the honey bee genome (Weinstock et al., 2006), quantitative trait locus analyses (Page, Rueppell, & Amdam, 2012), and transcriptomic studies (Zayed & Robinson, 2012) have positioned the honey bee at the forefront of efforts to understand the relationship between genes, the environment, and complex behavior. Adult worker honey bees exhibit behavioral maturation and transition between discrete sets of tasks as they age (G. E. Robinson, 1992). Bees perform tasks in the hive for the first 2-3 weeks of their 6-7 week adult life, such as cleaning or building new honeycomb and tending to (“nursing”) the brood. They then transition to working outside the hive, guarding its entrance or foraging for food and other resources. While this behavioral maturation has a strong age-related foundation, bees are also able to respond to changing colony conditions by accelerating, delaying or even reversing their trajectory. This behavioral plasticity is influenced by a complex of factors including genotypic background, colony demography, nutrition, and the availability of colony resources (G. E. Robinson, 1992). It is also mediated by specific endocrine factors and neuromodulators, and associated with changes in the expression of thousands of genes in the brain, some of which have causal effects on behavior (Zayed & Robinson, 2012). As a result, transcriptomic analyses of behavioral maturation in honey bees have led to fundamental insights about how genotype and the environment act on the brain transcriptome to regulate behavior (Zayed & Robinson, 2012).

Two particular behavioral states in honey bees, nursing and foraging, are often used to characterize the relationship between behavioral maturation and the transcriptome due to the well-characterized and distinct suites of behaviors that each entails. While the social, neuroendocrine, physiological, molecular, and genetic influences mediating these states have been elucidated in numerous studies, (Zayed & Robinson, 2012) the transcriptional regulatory architecture in the brain underlying and connecting these maturational determinants remains largely unknown. A brain transcriptional regulatory network (TRN) derived from co-expression data collected in a large set of microarray studies revealed that a small number of TFs were predicted to reliably regulate the vast majority of differentially expressed genes (DEGs) in the brain (Chandrasekaran et al., 2011). Similarly, examining the cis-regulatory logic underlying motifs present in the promoters and enhancers of DEGs revealed that specific combinations of motifs (many of which are binding sites for TFs identified in the above-mentioned brain TRN (Chandrasekaran et al., 2011)) were reliably associated with the differential expression of

105 maturation-related genes in the brain (S. A. Ament, Blatti, et al., 2012). Together, these results strongly suggest that a set of key TFs are responsive to maturational determinants and regulate definable gene modules to govern patterns of behavior. A comprehensive understanding of the manner in which these TFs contribute to behavioral state is thus essential to furthering our understanding of how behavior is organized.

As can be seen, there is great interest in elucidating the genome-scale TRNs underlying behavior (S. A. Ament, Blatti, et al., 2012; Chandrasekaran et al., 2011; Harris & Hofmann, 2014; O'Connell & Hofmann, 2011; Sanogo, Band, Blatti, Sinha, & Bell, 2012). However, because bioinformatics and experimental methods for identifying potential cis-regulatory sites upstream of the transcriptional start site (TSS) can be unreliable or difficult, respectively (Hardison & Taylor, 2012; Jeziorska, Jordan, & Vance, 2009), an ideal approach is to use a combination of methods to increase the robustness of inferences made about a network’s regulatory architecture. Since recent studies have highlighted the fact that a surprising proportion of potential binding sites in the promoter’s immediate vicinity exert functional influences on gene expression (T. W. Whitfield et al., 2012), the region surrounding the TSS may provide particularly valuable insights about the identity of the TFs regulating a gene. Indeed, it appears that TF binding at the promoter is so vital that regulator-target interactions during development can be conserved over vast evolutionary distances (Boyle et al., 2014).

Transcriptomic techniques based on cap analysis of gene expression (CAGE) allow for high- throughput deep sequencing of the 5’-ends of mRNA transcripts to identify a gene’s TSS as well as promoter features downstream of the start site by selectively enriching and sequencing the region immediately downstream of the 5’ methylguanosine cap (Harbers & Carninci, 2005). This allows one to spatially restrict motif finding to cis-regulatory modules that are actively co- transcribed with the target gene, and thus likely to be biologically relevant (Consortium, the, & Clst, 2014). These modules can then be used to create a high resolution map of the transcriptional start sites upstream of actively transcribed genes (Haberle et al., 2014).

In order to determine how TFs (as well as promoter and TSS characteristics) might contribute to behavior, we used CAGEscan (Plessy et al., 2010) to examine the transcriptional regulatory architecture in the brain underlying behavioral maturation. The large quantities of RNA required to perform traditional CAGE and SAGE techniques preclude the analysis of individual bee brains, a critical factor in accurately characterizing nuanced transcriptomic changes associated with

106 behavioral state. CAGEscan, however, is a variant of the nanoCAGE technique and is designed expressly for promoter characterization from small quantities of input RNA (Plessy et al., 2010). Mapping CAGEscan reads to a reference genome allows for accurate identification of TSS and the related promoter and 3’ region of the expressed gene. CAGEscan thus permits one to detect subtle changes in gene expression and link them to promoter characteristics such as motif composition of the promoter and TSS. With CAGEscan it is also possible to utilize paired-end- reads to provide additional information on the 3’end of the DNA fragments within the library. The additional 3’-end reads are used to improve mapping to the reference genome and to more accurately associate 5’-end reads to genes, and allowing for the discovery of novel promoter regions and TSSs. Here we report on the first comprehensive use of CAGEscan and mapping of TSS followed by promoter analysis of honey bee behavioral maturation.

Alternative TSSs are a pervasive feature in eukaryotic genomes, and a growing body of evidence indicates that they may play a vital role in gene regulation (de Hoon & Hayashizaki, 2008). While they can arise from distinct promoter regions clearly separated by long stretches of sequence, alternative TSSs can also occur close to each other within the same promoter region; even subtle alterations in a gene’s TSS have been associated with changes in the expression of downstream genes in Drosophila melanogaster (Brown et al., 2014; Hoskins et al., 2011) and mammals (Kawaji et al., 2006). CAGE-based techniques have already made valuable contributions to our understanding of transcription in model organisms such as, the fruit fly (Brown et al., 2014; Hoskins et al., 2011), zebrafish (Nepal et al., 2013), and human and mouse (Carninci, 2006; Consortium et al., 2014; Gustincich et al., 2006), and specifically in the nervous system, where alternative TSSs appear to play a role in establishing developmental (Pal et al., 2011) and region- specific (Pardo et al., 2013) gene expression patterns. However, the potential relevance of alternative TSSs in organizing behavior has, to our knowledge, not been addressed in any organism. A previous characterization of promoter usage at the transcriptome level using 5’ LongSAGE and expressed sequence tags found that there was evidence for TSS variability in nearly half of the genes transcribed in the head of male bees (Zheng et al., 2011), suggesting that promoter and TSS usage may also play a vital role in the regulatory systems underlying behavioral maturation.

Using CAGEscan to associate differentially expressed TFs with motif enrichment in the promoter region of DEGs, we were able to infer the identity of putative regulators of DEGs in specific behavioral contexts. Moreover, the identification of many of these TFs in previous analyses

107

(Chandrasekaran et al., 2011) suggests that they may play a role as regulators of not only individual genes, but of the behavioral state itself. If so, they would represent crucial links between the transcriptomic architecture and behavior. Finally, we used CAGEscan to accurately detect TSSs for every expressed gene, which enabled us to discover differences in TSS usage in different behavioral contexts. For the first time, this implicates alternative TSS usage as a potential mechanism regulating the transcriptomic changes underlying behavioral maturation.

2.2 Results and Discussion 2.2.1 Read Mapping and Gene Expression To elucidate the regulatory networks and TFs underlying behavioral plasticity in honey bees, we prepared CAGEscan libraries from the brains of individual nurses and foragers. Libraries were pooled into two groups of eight (corresponding to nurses and foragers) for sequencing on an Illumina platform (Figure 2.1), and sample-specific barcodes were used to differentiate between individuals. Initial sequencing of the forager samples revealed a low number of reads relative to standard RNAseq protocols. Since this deficiency in reads was likely due to the sequencing protocol rather than the quality of the RNA (Appendix Table B.1), the input cDNA of the nurse samples was increased to compensate. A total of 102,568,069 and 67,921,806 paired-reads were obtained from the sequencing of the nurse and forager samples, respectively; after filtering for read quality, 92,603,096 and 39,946,689 paired-reads were retained (Figure 2.1, Appendix Table B.2). 63% and 59% of nurse and forager reads, respectively, could be mapped to v4.5 of the honey bee reference genome (Appendix Table B.3), and were then processed for mapping quality (Appendix Table B.4). 83% to 90% of the CAGE tags from each sample could be mapped to genes in the honey bee genome (Appendix Table B.5), and we were able to associate CAGE tags with 13,111 genes. After normalizing and filtering the genes (see Methods), 12,453 of the 15,314 genes in OGSv3.2 (81.3%) had measurable levels of expression (Table 2.1). Despite the low quantity of read counts in our samples relative to traditional RNAseq studies, plotting saturation curves indicated that the degree of coverage was adequate to capture genes with a low level of expression, even in the forager samples (Appendix Figure A.1). For additional measures of read quality and distribution, see Appendix Tables B.1-B.5.

Comparing the per sample biological coefficient of variation (Appendix Figure A.2) and per gene squared coefficient of variation (Figure 2.2) revealed that there was a substantially higher degree of within-group variation in gene expression among foragers than nurses (p-value < 1.0e-300, Wilcoxon Rank Sum Test). Although this increase in variance could theoretically be due to the

108 lack of read coverage in forager samples relative to nurses (Appendix Table B.2), we minimized the impact of coverage-related biases by normalizing gene expression (see Methods). Moreover, if the variance was a result of low coverage, one would expect genes with a low level of expression to be the most adversely affected, and thus have the highest variance. However, this does not appear to be the case (Figure 2.2), indicating that read count did not contribute significantly to variation in gene expression or, by extension, differential gene analyses. It is possible, then, that the discrepancy in variation is a biologically relevant phenomenon and may reflect the fact that the foragers have to respond to a far more diverse set of stimuli (samples were collected on their return trip) and adapt to more variable conditions (i.e., outside environment and varying floral conditions) than do the hive-bound nurses. Although no prior study has explicitly compared nurse and forager variability in gene expression, forager variability has itself been the focus of other studies, which found that differences in experience, motivational state and environmental exposure can lead to distinct neurotranscriptomic states (Claudia C. Lutz, Rodriguez-Zas, Fahrbach, & Robinson, 2012; Zayed & Robinson, 2012).

Despite the disparity in within-group variance between nurses and foragers, unsupervised hierarchical clustering (Figures 2.3A and 2.3B, Appendix Figures A.3 and A.4) was able to generate two distinct groups of gene expression profiles that correspond directly to the behavioral state of the sampled bee. Hierarchical clustering also revealed discrete within-group clustering of the samples, which may reflect differences in within-group genetic relatedness (despite an average degree of relatedness of 75%), age, or time spent performing a particular activity (Claudia C. Lutz et al., 2012). A single outlier (sample F41) was identified during this analysis. However, subsequently removing the outlier had little impact on downstream analyses (Appendix Table B.6), and the sample was retained. Overall, these results indicate that CAGEscan was able to recapitulate the strong relationship between neurotranscriptomic and behavioral state observed in previous honey bee microarray studies (Alaux et al., 2009; C. W. Whitfield et al., 2006; C. W. Whitfield, Cziko, & Robinson, 2003).

2.2.2 Differentially Expressed Genes There were 1,058 differentially expressed genes (DEGs) between nurses and foragers (FDR < 0.05, Appendix Dataset C.1, Appendix Figure A.5). Although the number of DEGs upregulated in both groups is almost identical (534/524 genes in foragers and nurses, respectively), K-Means clustering revealed 29 clusters of upregulated genes in foragers and 21 clusters in nurses (Figure 2.4). This suggests that foragers may have greater variation in regulatory patterns, which is

109 consistent with our previous observations on the distribution of variance within the two behavioral groups.

The honey bee brain is surrounded by the hypopharyngeal glands (HPG), making it difficult to dissect the brain without the risk of contamination. Further, because the development of the HPG is intrinsically linked to the maturational state of the bee, contamination can result in systematic biases in gene expression when behavioral maturation is being assessed. Therefore, in order to determine the extent of potential contamination we used RNAseq to obtain an expression profile of nurse and forager HPGs relative to brain tissue. We then compared genes that were upregulated in the HPG to our dataset (Appendix Table B.7). Only 36 of the 1125 genes that were strongly (log2 fold-change > 3) upregulated in the HPG were identified as differentially expressed between nurses and foragers, implying that HPG contamination most likely had a minimal impact on the identification of DEGs. Since the potential influence of this contamination appeared to be negligible, no DEGs were removed from subsequent analyses.

2.2.3 Comparisons with Previous Studies To explore the concordance of these results with previous studies, we compared our data to prior microarray assessments of nurse and forager brain transcriptomes. For consistency, we remapped the microarray datasets to the current official honey bee gene set, OGSv3.2 using BLAT and Bowtie (Figure 2.5a). The present CAGEscan and previously published microarray datasets (Alaux et al., 2009; C. W. Whitfield et al., 2003) show strong similarities in the number of DEGs detected in the brain, with circa 800-900 DEGs for each study (Figure 2.5b, Appendix Dataset C.2). Moreover, the DEGs identified in the CAGEscan dataset exhibits a significant degree of overlap with prior microarray assessments of nurse and forager transcriptomes, sharing approximately 150 genes with each previous study (Figure 2.5b). Hypergeometric tests indicated that the degree of overlap between the three datasets was modest, but significant (p < 1e-08 for all pairwise comparisons, Bonferroni adjusted). The directional concordance of gene expression changes in the overlapping DEGs was highly consistent, however, with a minimum of 84% concordance (Figures 2.5c and 2.5d). Moreover, we calculated the Spearman Rank Correlation

(r) of the log2 fold change of our data and the aforementioned studies, and found robust and reliable correlations in gene expression values between the three studies (r=0.39, p < 1e-100, comparison of (Alaux et al., 2009; C. W. Whitfield et al., 2003); r=0.39, p < 1e-120, comparison of our results and (Alaux et al., 2009); r=0.25, p < 1e-125, comparison of our results and (C. W. Whitfield et al., 2003)).

110

These results are noteworthy given the differences in sample genetic background, collection protocol, analytical platforms and gene models used in these studies. In particular, models of alternative splicing are not as complete in the honey bee as they are in genetic model organisms, and have shifted considerably with the advent of newer annotations(Elsik et al., 2014). This could cause isoform specific probes to be misconstrued as indicating a change in overall gene expression when none actually exist. Finally, it should be noted that the degree of concordance between our study and the two array studies was not substantially different from the level of similarity between the two microarray studies themselves, suggesting that discrepancies between these studies may be the result of genetic background or biological noise rather than platform- related differences.

2.2.4 Gene Ontology Analyses of Differentially Expressed Genes A Gene Ontology (GO) analysis was performed to explore the functional implications of nurse and forager upregulated genes. Genes upregulated in nurses were found to be enriched for GO terms associated with nucleic acid, lipid and protein metabolism (Appendix Dataset C.3), a result consistent with previous transcriptomic analyses of behavioral maturation (Alaux et al., 2009). For instance, energy metabolism (S. A. Ament et al., 2011), oxidoreductase activity (C. W. Whitfield et al., 2006), oxidation reduction (S. A. Ament, Blatti, et al., 2012), glycolysis (S. A. Ament, Blatti, et al., 2012), and various mitochondrial and ribosomal (Naeger et al., 2011) components are all GO categories that were identified in both our study and previous studies on maturational determinants (Appendix Dataset C.4). These annotations are particularly relevant, since it is now well established that nutritional physiology has a causal influence on the behavioral state of the honey bee (Page et al., 2012; Zayed & Robinson, 2012). Manipulating factors that influence metabolic state such as diet (S. A. Ament et al., 2011), insulin signaling (Seth A. Ament, Corona, Pollock, & Robinson, 2008) and the yolk-protein Vitellogenin affect not only brain gene expression but the rate of behavioral maturation (Zayed & Robinson, 2012). Indeed, there is evidence of coordinated TRNs in honey bee brain and fat tissues during behavioral maturation, suggesting that brain function and body-wide metabolic changes are intrinsically linked at the level of the transcriptome (S. A. Ament, Wang, et al., 2012).

Genes upregulated in foragers were also enriched for some metabolic processes, but there was also far greater diversity in the types of GO terms that characterize forager up-regulated genes, including numerous terms associated with organ development and growth (Appendix Dataset

111

C.3). A closer inspection of these categories reveals that they are composed of genes known to play roles in nervous system development, neuronal function and neural plasticity in Drosophila melanogaster (Appendix Dataset C.4). As with nurses, the GO categories linked to foraging are also consistent with previous transcriptomic and informatics based analyses, especially for nervous system development (S. A. Ament, Blatti, et al., 2012; Sinha, Ling, Whitfield, Zhai, & Robinson, 2006), synaptic/neurotransmission (Chandrasekaran et al., 2011), receptor signaling pathways (C. W. Whitfield et al., 2006), protein kinase activity (Claudia C. Lutz et al., 2012; C. W. Whitfield et al., 2006), G-protein coupled receptor signaling (Grozinger, Sharabash, Whitfield, & Robinson, 2003; Claudia C. Lutz et al., 2012), insulin receptor signaling (Naeger et al., 2011), protein folding (S. A. Ament, Blatti, et al., 2012; Claudia C. Lutz et al., 2012; C. W. Whitfield et al., 2006), and response to heat (S. A. Ament, Blatti, et al., 2012; Claudia C. Lutz et al., 2012). These results may reflect the highly demanding cognitive tasks that foraging honey bees must perform relative to nurses related to navigation, manipulating flowers, and forming spatiotemporal memories of different foraging sites (Naeger et al., 2011), though experiments that directly manipulate the effects of these factors on the performance of foraging activities are still limited.

2.2.5 Transcription Factors Identified as Key Regulators of Behavioral Maturation In total, 250 orthologous TFs were identified by sequence similarity. 26 of these TFs were differentially expressed, with 4 upregulated in nurses and 22 upregulated in foragers (Appendix Dataset C.5, Appendix Figures A.6 and A.7). Additionally, more than half of the differentially expressed TFs had DNA binding motifs that were statistically enriched in the promoter regions of differentially expressed genes (Table 2.2), strongly suggesting they are part of the regulatory architecture underlying behavioral state.

Previous studies have indicated that the G/C content of promoter regions can have a dramatic impact on motif identification (Sinha et al., 2006). To ascertain whether our analysis was influenced by this bias, we compared the relative G/C content of promoters associated with forager and nurse upregulated genes. We found that the promoters of forager upregulated genes were indeed significantly enriched for G/C nucleotides compared to those of nurse upregulated genes (p-value < 1.0e-50, Wilcoxon Rank Sum Test, Appendix Dataset C.6, Figure 2.6). Since our initial analysis used nurse and forager promoters as background sets when assessing enrichment, a difference in C/G content between these groups could adversely affect these findings. In order to verify that our motif enrichment data were not compromised, we performed two additional analyses using alternative backgrounds consisting of 1) all predicted promoters in

112

OGS v3.2 or 2) randomized portions of the bee genome. Since the motifs of only two TFs were altered in these new analyses (Table 2.3), we conclude that C/G bias we detected exerted a minimal influence on our analysis.

To determine whether each of the 15 putative regulators of behavioral state might serve as activators or repressors of their target genes, we compared the expression patterns of the TFs themselves with the patterns of the genes they were predicted to regulate. Eight putative regulators had motifs that were enriched in the promoters of genes upregulated in the same behavioral context (Table 2.3) suggesting that they have an activating influence on their targets. Conversely, five putative regulators have a reciprocal relationship with their predicted targets, suggesting that they are serving as repressors of these genes. Finally, the last two TFs had motifs that were enriched in the promoters of both forager and nurse upregulated genes relative to all annotated promoters in the genome, suggesting they may have bivalent regulatory functions. Remarkably, these predictions are largely consistent with the known functions of orthologous genes in other organisms and contexts (Table 2.3). That being said, it should be noted that these functions may not correspond with canonical descriptions of the TF in question, as some of these TFs have been documented to possess dual activator and repressors functions in different contexts.

Two of these putative regulators, Creb1 and NF-κB, have previously been identified as potential regulators of behavioral maturation in both a reconstruction of the honey bee brain TRN (Chandrasekaran et al., 2011) and in motif distribution analyses of the regulatory regions of genes associated with behavioral maturation (S. A. Ament, Blatti, et al., 2012). Both Creb1 and NF-κB have also been experimentally shown to play vital roles in regulating neural plasticity (Barco & Marie, 2011; Benito & Barco, 2010; Meffert & Baltimore, 2005; Meffert, Chang, Wiltgen, Fanselow, & Baltimore, 2003), in addition to their involvement in other biological processes. Intriguingly, the genes of several TFs that interact with Creb1 were found to be differentially expressed, including atf3 and usf1. Like Creb1, Atf3 is a critical component of protein kinase A signaling (Chu, Tan, Kobierski, Balsam, & Comb, 1994), heterodimerizing with Creb1 to modulate gene expression in vertebrates. Indeed, according to our data, both Atf3 and Creb1 appear to be involved in instituting or maintaining the foraging state (Table 2.3), suggesting that they might be acting in a cooperative manner in bees as well. Usf1, by contrast, is known to work in opposition to Creb1 signaling (Steiger, Bandyopadhyay, Farb, & Russek, 2004) and is similarly predicted by our data to repress forager-related transcripts, potentially countering Creb1’s predicted role as an activator of foraging

113 related genes. Several modulators of NF-κB activity, including egr (Parra, Ferreira, & Ortega, 2011) (another putative regulator detailed below) and nr4a2 (which interacts with NF-κB in the nervous system (Saijo et al., 2009)) were also found to be differentially expressed in foragers. Together, these groups of genes may represent coherent regulatory modules governing behavioral state. At the very least, the fact that so many TFs known to interact with one another are predicted to regulate the same behavioral state reinforces the idea that the cellular functions regulated by Creb1 and NF-κB are particularly vital for the onset or maintenance of foraging behavior.

EGR, a TF that has previously been characterized as a canonical immediate early gene (IEG) linked to induction of neural plasticity in a variety of organisms (Knapska & Kaczmarek, 2004), was also upregulated in foragers. egr expression in the honey bee mushroom bodies (a region of the insect brain involved in learning and memory) is responsive to stimuli that trigger spatial learning (namely orientation flight) in conjunction with exposure to a novel environment (C. C. Lutz & Robinson, 2013). Quantitative PCR analyses additionally indicate that mushroom body egr expression increases in association with behavioral maturation independent of environmental stimuli (C. C. Lutz & Robinson, 2013). Our results concerning egr are therefore consistent with previous findings. Moreover, since the egr motif is enriched in the promoters of forager upregulated genes, these data suggest that egr functions not only as an IEG that governs transcriptomic responses to experiential stimuli, but also helps orchestrate the neurotranscriptomic changes that precede and maintain the foraging state as well.

The gene rxra1 (ultraspiracle/usp) is a highly conserved nuclear receptor with affinity for both juvenile hormone (Jones & Sharp, 1997) and ecdysone. Its identification as a putative regulator of foraging behavior is fitting, since endocrine signals (including juvenile hormone) are known to play a critical role in regulating behavioral maturation in honey bees (Corona et al., 2007). Moreover, experimental usp knockdown was previously shown to delay the transition to the foraging state (S. A. Ament, Wang, et al., 2012). This indicates that CAGEscan can “reproduce” known causal effects of genes on behavioral state, something that approaches based purely on informatics-derived inferences have sometimes failed to capture (Chandrasekaran et al., 2011). Intriguingly, the gene for Ecdysone Receptor (EcR), a binding partner of USP (Bitra & Palli, 2009), was also upregulated in foragers. While ecdysone has no known role in honey bee behavioral maturation, the co-expression of ecdysone receptor with its binding partner usp provides a suggestive hint that such a relationship exists, but has hitherto gone undetected.

114

The identification of clockwork orange (cwo), a critical component of the circadian regulatory circuit in Drosophila melanogaster (Kadener, Stoleru, McDonald, Nawathean, & Rosbash, 2007), as a putative regulator of behavioral maturation is also noteworthy. Although adult honey bees appear to possess endogenous biological rhythmicity from the moment they emerge from their cells, their locomotor behavior and metabolism are largely arrhythmic until shortly before the onset of foraging (Moore, 2001). Correspondingly, circadian related gene expression begins at a low and relatively invariant level, gradually increasing and becoming rhythmic as the bee approaches the foraging state. Additionally, the ability to form time-dependent memory is critical for honey bees, since they forage on resources that are both spatially and temporally restricted. Not only must a forager remember where a previously visited floral patch is, it must know when a floral patch is producing nectar and pollen. A previous study assaying brain gene expression changes in foragers found that the expression of genes associated with circadian rhythmicity not only cycle as a result of the time of day, but can also be modulated by training a bee to forage at a particular time point, suggesting they play a critical, perhaps even causal, role in organizing the temporal aspects of a bee’s foraging behavior (Naeger et al., 2011). Since all nurse and forager samples were collected within a very short time window (less than 1.5 hours), variation in cwo levels due to time of day should be minimal, suggesting that this gene may instead be serving a crucial function in the onset of spatiotemporal learning in honey bee foragers.

Finally, several TFs associated with nervous system development in Drosophila were also identified as putative regulators of the foraging state, namely: hes1 (in flies known as hairy or deadpan), dri (retained), pax6 (eyeless), hoxA6 (deformed), and hoxA1 (labial). Additionally, motifs associated with two of these TFs (dri and hairy) have previously been identified as enriched in the promoter regions of genes that are associated with behavioral maturation (Sinha et al., 2006). Neural plasticity associated with behavioral maturation in honey bees is known to involve large increases in dendritic arborization in specific brain regions (Farris, Robinson, & Fahrbach, 2001), and the cooption of developmental transcriptional programs may be one way this plasticity is mediated.

Remarkably, motifs associated with five differentially expressed transcription factors (hes1 (hairy), pax6, NF-κB, egr and clockwork orange) were combinatorially enriched in the promoters of nearly 50% of the genes that were upregulated in foragers (Appendix Figures A.8 and A.9, Appendix Dataset C.7). This suggests that a large proportion of the brain transcriptomic differences

115 between nurses and foragers may be influenced by a small number of TFs, a pattern that also has been predicted by previous bioinformatic analyses (Chandrasekaran et al., 2011). Moreover, the fact that such a large number of motifs were enriched in the same set of promoters implies that these five genes may co-regulate a coherent module of the regulatory architecture underlying behavioral state. It should be noted that the motif associated with one of these TFs (pax6) did not have the same level of enrichment when all OGS v3.2 promoters were used as the set of background sequences, suggesting that G/C bias may have had an influence in the detection of this particular motif (Table 2.3). Regardless, even if only the other four TFs are considered as putative co-regulators of such a significant proportion of the forager transcriptome, this is still a remarkable finding.

By contrast, only a single differentially expressed transcription factor, MyoD (nautilus), was associated with a motif enriched in more than 50% of the nurse upregulated genes. Traditionally known as a master regulator of cell fate in muscle cells (Tapscott, 2005), MyoD has only recently been characterized in the nervous system, where its only known function is as a tumor suppressor in the cerebellum of vertebrates (Dey et al., 2013). As such, this is the first discovery of the potential involvement of a MyoD ortholog as a key regulator of behavioral state, and elucidating its role in the insect nervous system will require additional study.

2.2.6 Alternative Transcriptional Start Sites and Behavioral Maturation In order to determine whether alternative TSSs were associated with behavioral state, we analyzed their occurrence in nurse and forager upregulated genes. For our purpose, TSSs are defined as the CAGE cluster with the highest degree of coverage (i.e., the most transcribed) that is common to all samples within a group (Figure 2.7). We first identified genes with multiple CAGE clusters across samples (Appendix Figure A.10), and then compared the results of our TSS analysis at each of these loci to determine whether there were systematic differences in TSS usage between foragers and nurses. Differential TSS usage was defined as the existence of distinct common TSSs in nurse and forager samples separated by a mutual distance of at least 100bp (Figure 2.7).

Our data indicate that 646 out of the 12,453 expressed genes possessed alternative TSSs that were utilized differentially between nurses and foragers (Appendix Dataset C.8). However, only 14.9% (96/646) of these genes were also found to be differentially expressed between nurses and foragers (Appendix Dataset C.8). One potential interpretation of this small proportion is that,

116 if alternative TSS selection plays a substantial role in regulating behavioral maturation in the honey bee, it does so by mediating splicing or post-transcriptional regulation of the resulting transcripts rather than directly influencing the levels of transcript produced. Alternative TSSs have been shown to have a significant effect on isoform expression (through differential recruitment of splicing factors or the exclusion of 5’ exons) (Carninci et al., 2006), mRNA turnover, and the efficiency of translation (Davuluri, Suzuki, Sugano, Plass, & Huang, 2008) in other species, so it is reasonable to speculate that they serve a variety of similar functions in the honey bee as well. Still, although the overlap between DEGs and alternative TSSs is small, the prevalence of genes with alternative TSSs is significantly higher in DEGs than in the whole transcriptome (Fisher's right-hand exact test using hypergeometric distribution. p-value < 3e-08). As such, it’s still possible that alternate TSSs play at least some role in regulating the rate of transcription during behavioral maturation.

Additionally, the small number of identified alternative TSSs relative to previous studies is related, in part, to our use of highly stringent criteria for the identification of TSSs. While the beginning of each CAGE tag can be considered as a discrete TSS, clustering CAGE tags is necessary to avoid false TSSs (Shiraki et al., 2003). Moreover, since we were interested in delineating the systematic differences in TSS between nurses and foragers, we clustered all CAGE tags within a 50bp window to determine a consensus start site for each group of bees. This provided a much more coherent picture of the distinct trends in start site selection between these two groups. In order to prevent tags from overlapping consensus sites, we further required that each alternative start site be separated by a mutual distance of at least 100bp. Relaxing either of these constraints dramatically increases the number of genes exhibiting alternative TSSs (Appendix Figure A.11). One should therefore consider the 646 genes with alternative TSSs to be a very conservative estimate of the link between behavioral state and TSS selection in the bee. Regardless, these results provide the first evidence that alternative TSSs reflect transcriptomic changes that are associated with sustained differences in behavior.

Gene Ontology (GO) analysis of the 646 genes with alternative TSSs show enrichment for a set of GO terms associated with nervous system development, neuronal development, axon guidance, wing development, oxidoreductase activity, lipid biosynthesis process and respiratory system development (Appendix Dataset C.9). These terms are strikingly similar to those obtained by GO analyses of DEGs (despite the low prevalence of DEGs exhibiting differential TSS usage)

117 and are strongly suggestive of a role for alternative TSS usage in establishing and/or maintaining differences in nervous system function between nurses and foragers.

2.3 Conclusions For the first time, we experimentally determined the TSSs and transcribed promoter regions associated with the regulation of behavioral plasticity in bees. We showed that the promoters of DEGs are enriched for motifs associated with many of the TFs we found to be differentially expressed, highlighting the potential importance of these TFs in regulating behavior. The coherent picture presented by our data and previous experimental and bioinformatics results reveals that CAGEscan provided us with highly detailed and convincing evidence about the functional architecture underlying the transcriptome during behavioral maturation. For instance, a number of these TFs were previously predicted to regulate behavioral maturation, and nearly all of them are associated with functions that correspond to known aspects of behavioral maturation.

Additionally, we found that a small subset of these putative regulators of behavioral state might be responsible for organizing the majority of transcriptomic differences in nurses and foragers, a result that corresponds with previous regulatory network analyses (Chandrasekaran et al., 2011). These results contribute to a growing appreciation of the fact that many behavioral states are associated with (and presumably regulated by) extensive and distinct transcriptional signatures in the brain (Drnevich et al., 2012; Zayed & Robinson, 2012). However, how such changes in RNA abundance lead to changes in neuronal function and, subsequently, behavior is a challenge that remains to be solved.

The fact that motif enrichment was assessed in actively transcribed promoter regions makes it all the more likely that the enriched motifs serve a functionally relevant role (T. W. Whitfield et al., 2012) in the transcriptional regulation of behavioral state. This is supported by the number of putative regulators that have previously been implicated in controlling behavioral maturation (Table 2.4). Still, we must stress that our results are purely correlative. Future studies should attempt to assess the veracity of these predictions by making targeted manipulations of these TFs and ascertaining their effect on behavioral state and the expression of predicted target genes.

Additionally, while the ability to associate differential TF and target gene expression with motif enrichment in actively transcribed regions is strongly suggestive of regulatory function, one should not expect all of a transcription factor’s potential targets to be regulated in every context,

118 particularly since genes are not commonly under the control of a single TF. Therefore, additional experiments are required to study how the combinatorial interactions between these TFs affect the expression of each target gene and give rise to contextually specific patterns of gene expression. Our findings implicate five TFs as putative co-regulators in nearly half the genes that were upregulated in foragers, which implies that TF co-association at the promoter may play a vital role in instituting or maintaining behavioral state. Because such combinatorial interactions have previously been predicted to play important roles in governing behavioral maturation (S. A. Ament, Blatti, et al., 2012) and TF co-association at the promoter appears to drive evolutionarily conserved differences in contextually dependent gene expression during development (Boyle et al., 2014), dissecting these patterns of co-regulation using targeted manipulations of the putative regulators is a logical next step in elucidating how the brain transcriptome organizes behavior.

Similarly, the lack of motif enrichment for differentially expressed TFs should not be construed as evidence that they are not involved in the regulation of behavioral maturation, particularly since the assay used here cannot account for the potential presence of TF binding sites at enhancers distal to the gene promoter. Similarly, this limitation makes it likely that a significant number of real targets were not characterized by CAGEscan. Therefore, the analyses presented here should be used to motivate and inform future experiments to study physical occupancy of potential binding sites by the most promising TFs, as has been done previously for Ultraspiracle Protein (S. A. Ament, Wang, et al., 2012).

It should be noted that, unlike previous studies (Alaux et al., 2009; C. W. Whitfield et al., 2006), we did not control for the effect of age on gene expression. Since the transition from hive-bound to foraging tasks involves a developmental trajectory, this presents a potential confound for our findings. However, previous studies assessing the contribution of chronological age relative to other maturational determinants have found that age plays a relatively minor role in determining differences between nurse and forager brain transcriptomes (C. W. Whitfield et al., 2006). Moreover, age-related differences in brain gene expression are most apparent in early adult life, generally long prior to onset of nursing and foraging behavior (C. W. Whitfield et al., 2006). Our results also exhibit high concordance with the predictions of a meta-analysis that assessed the link between maturational determinants (other than age) and transcriptomic architecture (S. A. Ament, Wang, et al., 2012). As such, we feel that while the potential age differential between nurses and foragers is doubtless responsible for some alterations in gene expression, it is unlikely to affect our overall conclusions.

119

Finally, applying CAGEscan we were able to identify reliable differences in TSS selection related to behavioral state for the first time. The transcripts for a substantial number of genes exhibit start sites unique to nursing or foraging behavior, and GO analysis indicates that these are relevant to nervous system function. While alternative TSSs may be regulating transcriptional rates in a comparatively small proportion of these genes, it’s also possible that they are contributing to the expression of alternative isoforms or other post-transcriptional regulatory processes that may contribute to the regulation of behavioral plasticity.

2.4 Methods 2.4.1 Sample Collection All samples were collected from a single colony at the University of Illinois Bee Research Facility, Urbana, Illinois. Samples were the offspring of a queen inseminated with semen from a single drone, which (due to the haplodiploid genetics of the honey bee) results in worker offspring with 75% average genetic relatedness. Behavioral identification was according to standard methods (S. A. Ament, Wang, et al., 2012). Bees that were observed entering honeycomb cells containing larvae were identified as nurses, immediately collected using forceps, and frozen in liquid nitrogen. Bees returning to the colony with loads of pollen on their hind legs were identified as foragers, captured using soft forceps and immediately frozen in liquid nitrogen. All collections (N = 25 nurses and foragers) were performed within a 1.5 hour timespan (from 10:00 to 11:30 a.m.) on the same day (July 29th, 2011). After collection, bee heads were freeze dried and brains were dissected in 80% ethanol chilled using dry ice (Schulz & Robinson, 1999).

2.4.2 RNA extraction Total RNA from individual bee heads was prepared by homogenizing the brain tissue using a motorized pestle and extracting the RNA using TRIzol (Life Technologies, Carlsbad, California, USA) and RNeasy Mini spin columns (Qiaqen, Venlo, Limberg, Netherlands), as per manufacturer specifications. All samples were treated with DNase (Qiagen). Sample quality was confirmed using a Nanodrop (Thermo Scientific, Walthan, Massachusettes, USA) and Bioanalyzer (Agilent Technologies, Santa Clara, California, USA).

2.4.3 CAGEscan Library Construction

120

CAGEscan libraries were generated from total RNA preparations of individual bee brains (16 samples including 8 nurses and 8 foragers), and the barcoded cDNAs were pooled into two libraries for sequencing using established protocol (Salimullah, Sakai, Mizuho, Plessy, & Carninci, 2011) (Figure 2.1). This protocol was modified slightly to reduce the rRNA content of CAGEscan libraries and to improve the selection of true 5' ends by incubating the RNA in 5´-Phosphate- Dependent Exonuclease (Terminator, Epicentre, Madison, USA) to remove rRNA and truncated mRNAs. During cDNA synthesis a reverse-transcription primer and “template-switching” oligonucleotide with individual barcodes (Appendix Table B.8) plus specific sequences for template switching at the 5’ cap of mRNA were incorporated into the first strand cDNA by a reverse transcriptase. Since the primer and template-switching oligonucleotide added known sequences to the 5’ and 3’ ends of the cDNA, they could be used as templates for semi- suppressive PCR. Using this process, long strands of cDNA were selectively amplified to generate the second cDNA strand (molecules that were short or possessed the same adaptor sequences at their 5’ and 3’ ends self-hybridized prior to the PCR, precluding amplification). The length of cDNA fragments within the CAGEscan library preparations ranged from 200-700 bps.

2.4.4 Sequencing of CAGEscan Libraries Sequencing of CAGEscan libraries was performed by the W.M. Keck Center for Comparative and Functional Genomics (University of Illinois at Urbana-Champaign, Urbana, Illinois, USA). Nurse and forager samples were combined into separate pools, and sequenced in different lanes and sequencing runs. Upon sequencing the forager samples, the quantity of reads obtained was judged to be lower than desired, and additional input cDNA was used for the nurse samples. CAGEscan tags used in this study were paired-end reads of length 100 bp. Low quality and outlier reads were filtered out of the data sets using FASTQC (http://www.bioinformatics.bbsrc.ac.uk/projects/fastqc), and CAGE tags with missing or incorrect adapters were omitted. In the sequence trimming process we removed the adapter sequence (21 bp, Appendix Table B.8) from the first mate of the paired-end sequences, and correspondingly pruned part the second mate, such that both mates had equal lengths (79 bps).

2.4.5 Mapping and Filtering CAGE Tags The 79 bp paired-end reads obtained after trimming were aligned to the honey bee reference genome (version 4.5) using Bowtie2 v2.1.0 (Langmead & Salzberg, 2012) in order to calculate the estimated mean (588 bp) and standard deviation (767 bp) of the inner distance between mapped paired-end reads. These parameters were then used with the Tophat v2.0.8 (Kim et al.,

121

2013) splice junction mapper to improve our ability to align the reads to the reference genome, allowing for up to 2 mismatches and 2 gaps per read. For the CAGE tag filtering process, we filtered out mapped reads that had a relatively high probability (p > 0.01/ MAPQ < 20) of being mapped incorrectly. Paired reads also were excluded from further analyses when: 1) both mates mapped to alternate strands, 2) one mate was unmapped, 3) the mates mapped to different scaffolds or 4) there was an inner distance greater than (mean + standard deviation) of the estimated inner distance between paired reads.

2.4.6 Gene Expression The CAGE tags were mapped to the official honey bee gene set, OGSv3.2 (Elsik et al., 2014). A typical CAGE tag was considered to be associated with a gene if it intersected with the region that covers [-2000 bp, end of the gene], but may be restricted by the end of the upstream gene on the same strand. In these cases the tag was considered to be associated if it maps to the region [end of the upstream gene+1, end of the gene]. As such, it is possible for multiple CAGE tags to be associated with one gene, or one CAGE tag to span two adjacent genes. To insure that the mapped reads provided sufficient coverage for differential expression analyses, their distribution was plotted using RSeQC (Wang, Wang, & Li, 2012). We generated a gene expression data matrix using the association of tags and genes, where each row represents the expression levels for a gene and each column represents a nurse or forager samples. Only those genes that had non-zero expression level in at least two samples of any of the nurse/forager groups were used for downstream analyses. Using this matrix, we normalized gene expression by rescaling the number of tags from each sample to the minimum number of tags from across all samples to remove sequencing bias.

2.4.7 Gene Clustering Based on Expression To determine the differences in brain gene expression levels between nurses and foragers, we performed two-way unsupervised hierarchical clustering using MATLAB to cluster genes and samples using an inner squared distance (minimum variance) algorithm. The Euclidean distance metric was used to measure the distances between gene profiles (rows) and Pearson’s correlation coefficient was used to measure the distance between sample profiles (columns). To obtain a statistical measure of how the clustering preserves the actual dissimilarities between samples, an unsupervised evaluation of hierarchical clustering using cophenetic correlation coefficient (CPCC) was performed. The CPCC is defined as:

122

∑푖<푗(푥푖푗 − 푥)(푑푖푗 − 푑) 퐶푃퐶퐶 = 2 2 √∑푖<푗(푥푖푗 − 푥) ∑푖<푗(푑푖푗 − 푑)

th th where 푥푖푗 is the Euclidean distance between i and j observation and 푑푖푗 is the cophenetic distance, which is the height of the link that joins the two observations in the obtained clustering dendrogram; x and d are the averages of 푥푖푗 and 푑푖푗, respectively. CPCC is the linear correlation coefficient between the observed distances (dissimilarities) in the samples and the cophenetic distances obtained from the clustering. In our case the CPCC was 0.78, suggesting that the clustering was not a technical artifact but represents actual biological differences between samples.

2.4.8 Variability of Gene Expression We evaluated differences in brain gene expression between individual bees within the nurse and forager groups by calculating the per-gene variance in expression levels between the individuals within each group. The variance was calculated on scaled expression data using the Z-score, such that the expression values of each gene had a mean equal to zero and standard deviation equal to 1. To examine if the variation in gene expression between forager samples was significantly different from the variation between nurse samples, we used the Wilcoxon Rank-Sum test between the two vectors of variances. Finally, we compared the samples using the per sample biological coefficient of variation (the square root of the dispersion parameter for the 500 most variable genes) and the per gene squared coefficient of variation (CV2) (the squared ratio of the standard deviation of gene expression across all group samples to the group average gene expression).

2.4.9 TSS Identification, Differential TSS Usage and Promoter Extraction To define TSS positions, CAGE tags belonging to each sample were clustered using an iterative hierarchical clustering approach with Paraclu v9 (Frith et al., 2008) to form clusters covering regions of less than 50 bp (Figure 2.7). Clusters that were more than 50 bp in length or were represented by fewer than 5 tags after rescaling were removed. Clusters with a maximum density/baseline density ratio of less than 2 also were excluded (since the signal strength was likely insufficient to represent a real TSS), as were clusters that were merely components of a larger cluster. We used these CAGE clusters to identify potential gene TSSs for the nurse and the forager groups independently of one another. Because more than one CAGE cluster could

123 potentially be associated to a particular gene, we defined a gene’s TSS to be the starting position of the CAGE cluster that has the greatest overall number of CAGE tags and is present in all of the samples in a group. These sites therefore represent a set of common TSSs for the expressed genes in each of the groups. To determine whether there was differential TSS usage between nurses and foragers, we compared the common TSS for each group. Those genes with distinct TSSs for each group were judged to use alternative start sites as a consequence of behavioral state. Due to the potential overlap of paired end reads in adjacent CAGE clusters, only TSSs with a mutual distance >100bp were considered for this analysis.

Promoters were defined as regions covering [-2000 bp, 200 bp] relative to TSSs common within a group. The final promoter region was further constrained so that it did not overlap an upstream gene or exceed the stop codon of the downstream gene to which the promoter was associated. Despite this restriction on promoter length, 65% of all OGSv3.2 genes (and 75% of differentially expressed genes) still use the full promoter region. Only 19% of OGSv3.2 genes (and 13% of differentially expressed genes) have promoters of <1000 bp length, and 9% of OGSv3.2 genes (and 5% of differentially expressed genes) have promoters of <500 bp length.

2.4.10 Differentially Expressed Genes (DEGs) The brain gene expression profiles of eight nurses and eight foragers were determined from the raw count of the CAGE tags associated with the respective genes. We filtered genes with a low level of expression, keeping only those that had at least 1 tag per million reads in at least 2 samples. To remove sequencing bias due to coverage depth, gene expression data were normalized using the Trimmed Mean of M-values (TMM) method (M. D. Robinson & Oshlack, 2010). Differentially expressed genes were determined on a per gene basis. We always compared genes of the same length to find differences in expression between the samples of each group, and gene length had no influence on the results. This allowed us to normalize based purely on the distribution of reads across the genes using the TMM in edgeR (M. D. Robinson, McCarthy, & Smyth, 2010). Statistical analyses of gene expression data to identify DEGs were performed in edgeR using tagwise dispersion to estimate the variance within each gene. EdgeR’s implementation of Fisher’s Exact Test (which corrects for overdispersion and uses a negative binomial distribution) was then performed to evaluate differential expression, and the resulting p-values were adjusted for multiple comparison testing using the Benjamini- Hochberg false discovery rate (FDR < 0.05).

124

The honey bee brain is surrounded by a large exocrine organ called the hypopharyngeal gland (HPG), which presents a potential source of contamination. Moreover, the HPG’s size and level of activity varies substantially in nurses and foragers, making it possible for contamination to bias gene expression assays and increase Type I error. Since it is impossible to quantify potential contamination directly, previous studies of nurse-forager gene expression have excluded genes with a high level of expression in the HPG (Alaux et al., 2009). To determine whether this would be necessary for our data, we used RNAseq to quantify the expression of genes in the HPGs (relative to brain tissue) of nurses and foragers. The top 1%, 5%, 10%, and 20% (by log fold change) of genes upregulated in the HPG of each group were then compared to their respective CAGEscan DEGs to determine the level of overlap. Since contamination is far more likely in nurse samples, genes that were upregulated in forager HPGs but also in the top 10% of nurse HPG upregulated genes were excluded from the forager overlap analysis (if contamination had occurred, it would have resulted in the false identification of nurse, rather than forager, upregulated genes).

2.4.11 DEG Overlap with Previous Studies To demonstrate the validity of data derived from CAGEscan and to provide a coherent picture of the genes that are most consistently differentially expressed in the brain as a function of behavioral maturation, we compared our results with those reported in two previous studies (Alaux et al., 2009; C. W. Whitfield et al., 2003). Previous studies were performed using two independently designed microarrays: one (C. W. Whitfield et al., 2003) containing ~9,000 probes based on honey bee expressed sequence tag data that predated the sequencing of the honey bee genome (Array Express Accession #A-MEXP-36), and a second (Alaux et al., 2009) with ~13,000 probes derived from gene annotations (OGS 2.0) for Assembly 2.0 of the sequenced genome (Array Express Accession #A-MEXP-755). For consistency, these datasets were reanalyzed by mapping the microarray probes to the current official honey bee gene set, OGSv3.2 (Elsik et al., 2014) using BLAT and Bowtie. Probes that could not be mapped to a unique gene were not used for further analyses. The microarray data were then corrected for multiple comparisons using a FDR cutoff of 0.05. In instances where multiple differentially expressed probes mapped to the same gene, the probes invariably exhibited the same direction of expression change across experimental groups. Therefore, duplicate probes were ignored. The significance of the overlap between each gene list was calculated using hypergeometric tests in SAS v9.4 and adjusted for multiple comparisons using a Bonferroni post hoc correction.

125

2.4.12 Functional Annotation of DEGs Gene Ontology (GO) (Ashburner et al., 2000) terms for the DEGs were determined using orthology to the Drosophila melanogaster genome, resulting in a total of 4,999 GO terms. GO enrichment analysis was performed based on the frequency of terms associated with the forager/nurse DEG list relative to the genomic background (all genes that had detectable levels of expression) using Fisher’s exact test, followed by FDR correction for multiple testing (FDR < 0.05). Analyses were performed using DAVID (Huang da, Sherman, & Lempicki, 2009) and the category-frequency of enriched GOs was analyzed using CateGOrizer (Hu, Bao, & Reecy, 2008).

2.4.13 Identification of TFs and Motif Finding around TSSs After analyzing differential expression, we identified TFs with Position Weight Matrix (PWM) models available in other organisms. To do so, we composed a list of 1,402 TFs (and their isoforms) associated with 676 PWMs from three different sources. We used 1,000 Human TFs from HOCOMOCO v9 database (Kulakovskiy et al., 2013) associated with 426 PWMs, 217 Drosophila TFs from Flybase (Marygold et al., 2013) associated with 73 PWMs, and 185 insect TFs from TRANSFAC Professional ver. 2012.2 (Matys et al., 2006) associated with 177 PWMs (Table 2.5). Then we compared the protein sequences of these TFs to the 15,314 protein sequences of A. mellifera OGS3.2 using OrthoMCL (Li, Stoeckert, & Roos, 2003) to find orthologous TFs. To identify TFs that might be key regulators of the nursing and foraging behavioral states, we used Clover (Frith et al., 2004) to assess whether associated motifs were overrepresented in the promoters of genes that were upregulated in nurses and foragers; motifs with similarity scores greater than 6 and a significance level of p-value < 0.05 were considered to be enriched. TFs that were differentially expressed and were associated with motifs enriched in genes upregulated in nurses or foragers were considered to be putative regulators of those respective behavioral states.

A previous informatics analysis uncovered a systematic bias toward high Guanine/Cytosine (G/C) content in the promoters of genes upregulated in foragers (relative to nurse associated promoters)that led to an overestimate of the number of overrepresented TF motifs associated with behavioral state (Sinha et al., 2006). To ascertain whether a similar bias exists in the CAGE tags that comprise our dataset, we compared the ratio of G/C to A/T nucleotides in the reconstructed promoters of forager and nurse upregulated genes using the Wilcoxon Rank-Sum Test. We then accounted for differences in G/C content by performing our cis-motif enrichment

126 analysis using three different backgrounds. For the first test, the background consisted of promoters from genes upregulated in the behavioral state that was not being assessed (i.e., the promoters of forager upregulated genes used the promoters of nurse upregulated genes as a background) in order to emphasize the distinctions in motif distribution between these sets of promoters. We then performed two additional analyses to confirm the validity of these findings, using either: 1) all predicted promoters in OGS v3.2 or 2) randomized portions of the bee genome as the background for each set of promoters.

Acknowledgments: I would like to acknowledge and thank my coauthors Abdullah M. Khamis, Yulia A. Medvedeva, Tanvir Alam, Intikhab Alam, Magbubah Essack, Boris Umylny, Boris R. Jankovic, Nicholas L. Naeger, Makoto Suzuki, Matthias Harbers, Gene E. Robinson & Vladimir B. Bajic for their contributions toward this work. I’d also like to thank D.C. Nye for assistance with the bees; T.N. Newman for assistance with RNA extraction; Charles Plessy for assistance in developing the modified CAGEscan methods used in this paper. This work was supported by National Institutes of Health Director’s Pioneer Award 1DP1OD006416 (to G.E.R.). AMK, YAM, TA, IA, ME, BRJ and VBB were supported by KAUST research funds.

127

2.5 Figures

Figure 2.1. Overview of library preparation and sequencing. CAGEscan libraries were generated from total RNA extracted from individual brains of 8 nurses and 8 foragers. Barcoded cDNAs were pooled into two lanes and sequenced on an Illumina HiSeq2000. The resulting reads can be viewed in the table on the right.

128

Figure 2.2. The squared coefficient of variation (CV2) in per-gene expression for foragers and nurses. The x-axis is the log10 normalized per-gene expression level and the y-axis is the squared coefficient of variance (CV2). It is apparent that variability in gene expression within foragers is higher than nurses for most genes, yet not for genes with a low level of expression (which should be the most prone to variation arising from technical artifacts).

129

Figure 2.3. Hierarchical clustering of the brain gene expression profiles of nurse and forager honey bees. Clustering was performed using Ward’s method. Rows correspond to 100 clusters obtained from 12,453 genes by the k-means algorithm and columns represent nurse (‘N’) and forager (‘F’) samples. The scale bar indicates the z-scores of gene expression values, such that highly expressed genes are depicted in (dark red) while genes with low levels of expression are depicted in (dark blue). A heatmap showing the hierarchical clustering of all 12,453 genes without K-Means clustering is provided in (Supplementary Fig. S2).

130

Figure 2.4. Heatmap for the hierarchical clustering of the differentially expressed brain gene profiles of nurse and forager honey bees. Rows correspond to 50 clusters obtained from 1,058 DEGs by the k-means algorithm. Columns represent samples. The scale bar indicates z- scores of gene expression values, with highly expressed genes depicted in dark red low- expressed genes depicted in dark blue. The heatmap that shows the hierarchical clustering of all 1,058 DEGs without clustering is provided in (Appendix Figure A.3).

131

Figure 2.5. Overlap of DEGs between CAGEscan and previous studies of nurse and forager brain transcriptomes. a) Represents the relationship between gene models of the newest honey bee genome annotation (OGS 3.2) and the probes that were present on microarray platforms used in previous analyses of honey bee nursing and foraging behavior. Only probes that could be mapped to OGS 3.2 and genes that were present on at least one array (shown in the regions of overlap) were used to assess commonalities between CAGEscan and the two cited studies. 5b) Shows the overlap of differentially expressed genes detected by CAGEscan and the two previous microarray-based studies of nurse and forager transcription. 5c & 5d) These Venn diagrams display the degree of directional concordance for nurse (5c) and forager (5d) upregulated genes in the three studies. The areas of overlap represent the number of concordant genes, while the numbers in parentheses indicate the percentage of concordance relative to the number of differentially expressed genes associated with nursing and foraging in each study.

132

Figure 2.6. G/C content distribution for the promoters of DEGs. Promoters associated with Forager upregulated genes have a significantly higher percentage of G/C nucleotides than those associated with Nurse upregulated genes, which could bias motif enrichment analyses.

133

Figure 2.7. CAGE tags mapping, clustering and TSS identification. After sequencing, the CAGE tags were mapped to v3.2 of the honey bee Official Gene Set to form clusters (the 3’paired end reads are used to facilitate this mapping). A cluster was identified as a ‘common’ TSS for each group if it had the greatest number of CAGE tags relative to all other clusters and was present in all samples within the group. The location of the forager and nurse TSSs was then compared to determine whether differential TSS selection occurred as a consequence of behavioral state. Additionally, promoter regions identified using CAGE can be scanned for differences in TF binding site occurrence to gain insights into the regulatory architecture controlling each gene’s transcription.

134

2.6 Tables

Table 2.1. Number of genes that could be associated with CAGE tags. Number of

genes Total number of genes in OGS3.2 15,314 Number of genes associated with CAGE tags in at least one 13,111 (85.6%) sample Number of genes associated with CAGE tags in two or more 12,453 (81.3%) samples

135

Table 2.2. Putative Transcriptional Regulators of Behavioral State. Human Bee Gene Fly Ortholog Ortholog Identifier USF1/USF2 usf GB40634 Nursing CXXC1 cfp1 GB43820 Regulators MYOD1 nautilus GB55306 RXRA/RXRB ultraspiracle GB42692 CREB1 Creb-B17A GB46492 C/EBP slbo GB44204 DFD deformed GB51299 HXA1 labial GB51303 ATF3 atf3 GB53401 Foraging DRI retained GB55596 Regulators NF-κB dorsal GB42472 EGR1/EGR2 stripe GB50091 PAX6 eyeless GB50342 HES1 Hairy GB47799 BHE40/BHE cwo GB52039 41

136

Table 2.3. Motif Enrichment and Predicted Function of Differentially Expressed TFs in Promoters Associated with Nurse and Forager Upregulated Genes. Nurse DEG Enriched in Enriched in Predicted Predicted Promoters Nurse DEG Forager DEG Transcriptional Function TF Name (Relative to Promoters Promoters Activity Consistent with Forager DEG (Relative to all (Relative to all Known Roles? Promoters) Promoters) Promoters) Upregulated MYOD ↑ ↑ N.S. Activator Yes (Tapscott, 2005) in Nurses cfp1 ↓ N.S. ↑ Repressor No usf ↓ N.S. ↑ Repressor Yes (Steiger et al., 2004) Upregulated Activator & Yes (S. A. Ament, usp ↑ ↑ ↑ in Foragers Repressor Wang, et al., 2012) Activator & Yes (Hai & Hartman, atf3 ↑ ↑ ↑ Repressor 2001) ↑ ↑ N.S. Repressor Yes (Kowenz-Leutz, Twamley, Ansieau, C/EBP (slbo) & Leutz, 1994; Ramji & Foka, 2002) ↑ ↑ N.S. Repressor Yes (Pinsonneault, deformed Florence, Vaessin, & McGinnis, 1997) labial ↑ N.S. N.S. Repressor No ↓ Activator Yes (Shandala, retained N.S. ↑ Kortschak, Gregory, & Saint, 1999) ↓ N.S. ↑ Activator Yes (Steiger et al., Creb-B17A 2004) ↓ Activator Yes (Meffert & NF-κB (dorsal) N.S. ↑ Baltimore, 2005) ↓ Activator Yes (Knapska & EGR (stripe) N.S. ↑ Kaczmarek, 2004) ↓ N.S. N.S. Activator Yes (Weasner, Weasner, Deyoung, PAX6 (eyeless) Michaels, & Kumar, 2009) ↓ N.S. ↑ Activator Yes (Jimenez, HES1 (hairy) Pinchin, & Ish- Horowicz, 1996) ↓ N.S. ↑ Activator Yes (Richier, Michard-Vanhee, cwo Lamouroux, Papin, & Rouyer, 2008)

↑ – The motif(s) associated with the listed TF is/are overrepresented in promoters associated with genes upregulated in the given context, relative to the given background. ↓ – The motif(s) associated with the listed TF is/are not overrepresented in promoters associated with genes upregulated in the given context, relative to the given background (i.e. the motif(s) were overrepresented in the background context). N.S. – There was no significant difference in motif frequency between the promoters in the listed context and the background. TFs in (red) font had motifs that were not enriched in the same gene sets when analyzed with different backgrounds (i.e. may have been subject to C/G bias).

137

Table 2.4. Comparison of CAGEscan and previous analyses identifying putative regulators of behavioral state in the honey bee. Transcriptional Combinatorial Regulatory Cis-Motif (Sinha et Motif Analysis TF Name Network Analysis (S. A. Ament, Blatti, et (Chandrasekaran et al., 2006) al., 2012) al., 2011) usf Nursing cfp1 Regulators MYOD1 (nautilus) usp Creb-B17A C/EBP (slbo) Foraging retained Regulators atf3 labial deformed NF-κB (dorsal) Foraging hairy Key PAX6 (eyeless) Regulators EGR (stripe) cwo

Green - TFs predicted to regulate either nursing or foraging in both the cited analysis and the current study. Red - TFs that are predicted by CAGEscan but not the cited study to regulate behavioral maturation. Black - The TF of interest was not assessed in the cited analysis.

138

Table 2.5. Summary of the 676 PWMs associated with 1,402 TFs collected from TRANSFAC, Flybase and HOCOMOCO. Number of Number of TFs Number of TFs

PWMs Without Isoforms and Isoforms

TRANSFAC 177 133 185

Flybase 73 142 217

HOCOMOCO 426 405 1,000

Total 676 680 1,402

139

References Alaux, C., Le Conte, Y., Adams, H. A., Rodriguez-Zas, S., Grozinger, C. M., Sinha, S., & Robinson, G. E. (2009). Regulation of brain gene expression in honey bees by brood pheromone. Genes Brain Behav, 8(3), 309-319. doi:10.1111/j.1601-183X.2009.00480.x Ament, S. A., Blatti, C. A., Alaux, C., Wheeler, M. M., Toth, A. L., Le Conte, Y., . . . Sinha, S. (2012). New meta-analysis tools reveal common transcriptional regulatory basis for multiple determinants of behavior. Proc Natl Acad Sci U S A, 109(26), E1801-1810. doi:10.1073/pnas.1205283109 Ament, S. A., Chan, Q. W., Wheeler, M. M., Nixon, S. E., Johnson, S. P., Rodriguez-Zas, S. L., . . . Robinson, G. E. (2011). Mechanisms of stable lipid loss in a social insect. J Exp Biol, 214(Pt 22), 3808-3821. doi:10.1242/jeb.060244 Ament, S. A., Corona, M., Pollock, H. S., & Robinson, G. E. (2008). Insulin signaling is involved in the regulation of worker division of labor in honey bee colonies. Proc Natl Acad Sci U S A, 105(11), 4226-4231. doi:10.1073/pnas.0800630105 Ament, S. A., Wang, Y., Chen, C. C., Blatti, C. A., Hong, F., Liang, Z. S., . . . Robinson, G. E. (2012). The transcription factor ultraspiracle influences honey bee social behavior and behavior-related gene expression. PLoS Genet, 8(3), e1002596. doi:10.1371/journal.pgen.1002596 Ashburner, M., Ball, C. A., Blake, J. A., Botstein, D., Butler, H., Cherry, J. M., . . . Sherlock, G. (2000). Gene ontology: tool for the unification of biology. The Gene Ontology Consortium. Nat Genet, 25(1), 25-29. doi:10.1038/75556 Barco, A., & Marie, H. (2011). Genetic approaches to investigate the role of CREB in neuronal plasticity and memory. Mol Neurobiol, 44(3), 330-349. doi:10.1007/s12035-011-8209-x Benito, E., & Barco, A. (2010). CREB's control of intrinsic and synaptic plasticity: implications for CREB-dependent memory models. Trends Neurosci, 33(5), 230-240. doi:10.1016/j.tins.2010.02.001 Bitra, K., & Palli, S. R. (2009). Interaction of proteins involved in ecdysone and juvenile hormone signal transduction. Arch Insect Biochem Physiol, 70(2), 90-105. doi:10.1002/arch.20281 Boyle, A. P., Araya, C. L., Brdlik, C., Cayting, P., Cheng, C., Cheng, Y., . . . Snyder, M. (2014). Comparative analysis of regulatory information and circuits across distant species. Nature, 512(7515), 453-456. doi:10.1038/nature13668 Brown, J. B., Boley, N., Eisman, R., May, G. E., Stoiber, M. H., Duff, M. O., . . . Celniker, S. E. (2014). Diversity and dynamics of the Drosophila transcriptome. Nature, 512(7515), 393-

140

399. doi:10.1038/nature12962 Carninci, P. (2006). Tagging mammalian transcription complexity. Trends Genet, 22(9), 501- 510. doi:10.1016/j.tig.2006.07.003 Carninci, P., Sandelin, A., Lenhard, B., Katayama, S., Shimokawa, K., Ponjavic, J., . . . Hayashizaki, Y. (2006). Genome-wide analysis of mammalian promoter architecture and evolution. Nat Genet, 38(6), 626-635. doi:10.1038/ng1789 Chandrasekaran, S., Ament, S. A., Eddy, J. A., Rodriguez-Zas, S. L., Schatz, B. R., Price, N. D., & Robinson, G. E. (2011). Behavior-specific changes in transcriptional modules lead to distinct and predictable neurogenomic states. Proc Natl Acad Sci U S A, 108(44), 18020- 18025. doi:10.1073/pnas.1114093108 Chu, H. M., Tan, Y., Kobierski, L. A., Balsam, L. B., & Comb, M. J. (1994). Activating transcription factor-3 stimulates 3',5'-cyclic adenosine monophosphate-dependent gene expression. Mol Endocrinol, 8(1), 59-68. doi:10.1210/mend.8.1.8152431 Consortium, F., the, R. P., & Clst. (2014). A promoter-level mammalian expression atlas. Nature, 507(7493), 462-470. doi:10.1038/nature13182 Corona, M., Velarde, R. A., Remolina, S., Moran-Lauter, A., Wang, Y., Hughes, K. A., & Robinson, G. E. (2007). Vitellogenin, juvenile hormone, insulin signaling, and queen honey bee longevity. Proc Natl Acad Sci U S A, 104(17), 7128-7133. doi:10.1073/pnas.0701909104 Davuluri, R. V., Suzuki, Y., Sugano, S., Plass, C., & Huang, T. H. (2008). The functional consequences of alternative promoter use in mammalian genomes. Trends Genet, 24(4), 167-177. doi:10.1016/j.tig.2008.01.008 de Hoon, M., & Hayashizaki, Y. (2008). Deep cap analysis gene expression (CAGE): genome- wide identification of promoters, quantification of their expression, and network inference. Biotechniques, 44(5), 627-628, 630, 632. doi:10.2144/000112802 Dey, J., Dubuc, A. M., Pedro, K. D., Thirstrup, D., Mecham, B., Northcott, P. A., . . . Olson, J. M. (2013). MyoD is a tumor suppressor gene in medulloblastoma. Cancer Res, 73(22), 6828-6837. doi:10.1158/0008-5472.CAN-13-0730-T Drnevich, J., Replogle, K. L., Lovell, P., Hahn, T. P., Johnson, F., Mast, T. G., . . . Clayton, D. F. (2012). Impact of experience-dependent and -independent factors on gene expression in songbird brain. Proc Natl Acad Sci U S A, 109 Suppl 2, 17245-17252. doi:10.1073/pnas.1200655109 Elsik, C. G., Worley, K. C., Bennett, A. K., Beye, M., Camara, F., Childers, C. P., . . . Honey Bee Genome Sequencing, C. (2014). Finding the missing honey bee genes: lessons learned

141

from a genome upgrade. BMC Genomics, 15, 86. doi:10.1186/1471-2164-15-86 Farris, S. M., Robinson, G. E., & Fahrbach, S. E. (2001). Experience- and age-related outgrowth of intrinsic neurons in the mushroom bodies of the adult worker honeybee. J Neurosci, 21(16), 6395-6404. Frith, M. C., Fu, Y., Yu, L., Chen, J. F., Hansen, U., & Weng, Z. (2004). Detection of functional DNA motifs via statistical over-representation. Nucleic Acids Res, 32(4), 1372-1381. doi:10.1093/nar/gkh299 Frith, M. C., Valen, E., Krogh, A., Hayashizaki, Y., Carninci, P., & Sandelin, A. (2008). A code for transcription initiation in mammalian genomes. Genome Res, 18(1), 1-12. doi:10.1101/gr.6831208 Grozinger, C. M., Sharabash, N. M., Whitfield, C. W., & Robinson, G. E. (2003). Pheromone- mediated gene expression in the honey bee brain. Proc Natl Acad Sci U S A, 100 Suppl 2, 14519-14525. doi:10.1073/pnas.2335884100 Gustincich, S., Sandelin, A., Plessy, C., Katayama, S., Simone, R., Lazarevic, D., . . . Carninci, P. (2006). The complexity of the mammalian transcriptome. J Physiol, 575(Pt 2), 321- 332. doi:10.1113/jphysiol.2006.115568 Haberle, V., Li, N., Hadzhiev, Y., Plessy, C., Previti, C., Nepal, C., . . . Lenhard, B. (2014). Two independent transcription initiation codes overlap on vertebrate core promoters. Nature, 507(7492), 381-385. doi:10.1038/nature12974 Hai, T., & Hartman, M. G. (2001). The molecular biology and nomenclature of the activating transcription factor/cAMP responsive element binding family of transcription factors: activating transcription factor proteins and homeostasis. Gene, 273(1), 1-11. Harbers, M., & Carninci, P. (2005). Tag-based approaches for transcriptome research and genome annotation. Nat Methods, 2(7), 495-502. doi:10.1038/nmeth768 Hardison, R. C., & Taylor, J. (2012). Genomic approaches towards finding cis-regulatory modules in animals. Nat Rev Genet, 13(7), 469-483. doi:10.1038/nrg3242 Harris, R. M., & Hofmann, H. A. (2014). Neurogenomics of behavioral plasticity. Adv Exp Med Biol, 781, 149-168. doi:10.1007/978-94-007-7347-9_8 Hoskins, R. A., Landolin, J. M., Brown, J. B., Sandler, J. E., Takahashi, H., Lassmann, T., . . . Celniker, S. E. (2011). Genome-wide analysis of promoter architecture in Drosophila melanogaster. Genome Res, 21(2), 182-192. doi:10.1101/gr.112466.110 Hu, Z., Bao, J., & Reecy, J. M. (2008). CateGOrizer: A web-based program to batch analyze gene ontology classification categories. Onl J Bioinform, 9(2), 108-112. Huang da, W., Sherman, B. T., & Lempicki, R. A. (2009). Systematic and integrative analysis of

142

large gene lists using DAVID bioinformatics resources. Nat Protoc, 4(1), 44-57. doi:10.1038/nprot.2008.211 Jeziorska, D. M., Jordan, K. W., & Vance, K. W. (2009). A systems biology approach to understanding cis-regulatory module function. Semin Cell Dev Biol, 20(7), 856-862. doi:10.1016/j.semcdb.2009.07.007 Jimenez, G., Pinchin, S. M., & Ish-Horowicz, D. (1996). In vivo interactions of the Drosophila Hairy and Runt transcriptional repressors with target promoters. EMBO J, 15(24), 7088- 7098. Jones, G., & Sharp, P. A. (1997). Ultraspiracle: an invertebrate nuclear receptor for juvenile hormones. Proc Natl Acad Sci U S A, 94(25), 13499-13503. Kadener, S., Stoleru, D., McDonald, M., Nawathean, P., & Rosbash, M. (2007). Clockwork Orange is a transcriptional repressor and a new Drosophila circadian pacemaker component. Genes Dev, 21(13), 1675-1686. doi:10.1101/gad.1552607 Kawaji, H., Frith, M. C., Katayama, S., Sandelin, A., Kai, C., Kawai, J., . . . Hayashizaki, Y. (2006). Dynamic usage of transcription start sites within core promoters. Genome Biol, 7(12), R118. doi:10.1186/gb-2006-7-12-r118 Khamis, A. M., Hamilton, A. R., Medvedeva, Y. A., Alam, T., Alam, I., Essack, M., . . . Bajic, V. B. (2015). Insights into the Transcriptional Architecture of Behavioral Plasticity in the Honey Bee Apis mellifera. Scientific Reports, 5. doi:10.1038/srep11136 Kim, D., Pertea, G., Trapnell, C., Pimentel, H., Kelley, R., & Salzberg, S. L. (2013). TopHat2: accurate alignment of transcriptomes in the presence of insertions, deletions and gene fusions. Genome Biol, 14(4), R36. doi:10.1186/gb-2013-14-4-r36 Knapska, E., & Kaczmarek, L. (2004). A gene for neuronal plasticity in the mammalian brain: Zif268/Egr-1/NGFI-A/Krox-24/TIS8/ZENK? Prog Neurobiol, 74(4), 183-211. doi:10.1016/j.pneurobio.2004.05.007 Kowenz-Leutz, E., Twamley, G., Ansieau, S., & Leutz, A. (1994). Novel mechanism of C/EBP beta (NF-M) transcriptional control: activation through derepression. Genes Dev, 8(22), 2781-2791. Kulakovskiy, I. V., Medvedeva, Y. A., Schaefer, U., Kasianov, A. S., Vorontsov, I. E., Bajic, V. B., & Makeev, V. J. (2013). HOCOMOCO: a comprehensive collection of human transcription factor binding sites models. Nucleic Acids Res, 41(Database issue), D195- 202. doi:10.1093/nar/gks1089 Langmead, B., & Salzberg, S. L. (2012). Fast gapped-read alignment with Bowtie 2. Nat Methods, 9(4), 357-359. doi:10.1038/nmeth.1923

143

Li, L., Stoeckert, C. J., Jr., & Roos, D. S. (2003). OrthoMCL: identification of ortholog groups for eukaryotic genomes. Genome Res, 13(9), 2178-2189. doi:10.1101/gr.1224503 Lutz, C. C., & Robinson, G. E. (2013). Activity-dependent gene expression in honey bee mushroom bodies in response to orientation flight. J Exp Biol, 216(Pt 11), 2031-2038. doi:10.1242/jeb.084905 Lutz, C. C., Rodriguez-Zas, S. L., Fahrbach, S. E., & Robinson, G. E. (2012). Transcriptional response to foraging experience in the honey bee mushroom bodies. Developmental Neurobiology, 72(2), 153-166. doi:10.1002/dneu.20929 Marygold, S. J., Leyland, P. C., Seal, R. L., Goodman, J. L., Thurmond, J., Strelets, V. B., . . . FlyBase, c. (2013). FlyBase: improvements to the bibliography. Nucleic Acids Res, 41(Database issue), D751-757. doi:10.1093/nar/gks1024 Matys, V., Kel-Margoulis, O. V., Fricke, E., Liebich, I., Land, S., Barre-Dirrie, A., . . . Wingender, E. (2006). TRANSFAC and its module TRANSCompel: transcriptional gene regulation in eukaryotes. Nucleic Acids Res, 34(Database issue), D108-110. doi:10.1093/nar/gkj143 Meffert, M. K., & Baltimore, D. (2005). Physiological functions for brain NF-kappaB. Trends Neurosci, 28(1), 37-43. doi:10.1016/j.tins.2004.11.002 Meffert, M. K., Chang, J. M., Wiltgen, B. J., Fanselow, M. S., & Baltimore, D. (2003). NF-kappa B functions in synaptic signaling and behavior. Nat Neurosci, 6(10), 1072-1078. doi:10.1038/nn1110 Moore, D. (2001). Honey bee circadian clocks: behavioral control from individual workers to whole-colony rhythms. J Insect Physiol, 47(8), 843-857. doi:http://dx.doi.org/10.1016/S0022-1910(01)00057-9 Naeger, N. L., Van Nest, B. N., Johnson, J. N., Boyd, S. D., Southey, B. R., Rodriguez-Zas, S. L., . . . Robinson, G. E. (2011). Neurogenomic signatures of spatiotemporal memories in time-trained forager honey bees. J Exp Biol, 214(Pt 6), 979-987. doi:10.1242/jeb.053421 Nepal, C., Hadzhiev, Y., Previti, C., Haberle, V., Li, N., Takahashi, H., . . . Muller, F. (2013). Dynamic regulation of the transcription initiation landscape at single nucleotide resolution during vertebrate embryogenesis. Genome Res, 23(11), 1938-1950. doi:10.1101/gr.153692.112 O'Connell, L. A., & Hofmann, H. A. (2011). Genes, hormones, and circuits: an integrative approach to study the evolution of social behavior. Front Neuroendocrinol, 32(3), 320- 335. doi:10.1016/j.yfrne.2010.12.004 Page, R. E., Jr., Rueppell, O., & Amdam, G. V. (2012). Genetics of reproduction and regulation of honeybee (Apis mellifera L.) social behavior. Annu Rev Genet, 46, 97-119.

144

doi:10.1146/annurev-genet-110711-155610 Pal, S., Gupta, R., Kim, H., Wickramasinghe, P., Baubet, V., Showe, L. C., . . . Davuluri, R. V. (2011). Alternative transcription exceeds alternative splicing in generating the transcriptome diversity of cerebellar development. Genome Res, 21(8), 1260-1272. doi:10.1101/gr.120535.111 Pardo, L. M., Rizzu, P., Francescatto, M., Vitezic, M., Leday, G. G., Sanchez, J. S., . . . Heutink, P. (2013). Regional differences in gene expression and promoter usage in aged human brains. Neurobiol Aging, 34(7), 1825-1836. doi:10.1016/j.neurobiolaging.2013.01.005 Parra, E., Ferreira, J., & Ortega, A. (2011). Overexpression of EGR-1 modulates the activity of NF-kappaB and AP-1 in prostate carcinoma PC-3 and LNCaP cell lines. Int J Oncol, 39(2), 345-352. doi:10.3892/ijo.2011.1047 Pinsonneault, J., Florence, B., Vaessin, H., & McGinnis, W. (1997). A model for extradenticle function as a switch that changes HOX proteins from repressors to activators. EMBO J, 16(8), 2032-2042. doi:10.1093/emboj/16.8.2032 Plessy, C., Bertin, N., Takahashi, H., Simone, R., Salimullah, M., Lassmann, T., . . . Carninci, P. (2010). Linking promoters to functional transcripts in small samples with nanoCAGE and CAGEscan. Nat Methods, 7(7), 528-534. doi:10.1038/nmeth.1470 Ramji, D. P., & Foka, P. (2002). CCAAT/enhancer-binding proteins: structure, function and regulation. Biochem J, 365(Pt 3), 561-575. doi:10.1042/BJ20020508 Richier, B., Michard-Vanhee, C., Lamouroux, A., Papin, C., & Rouyer, F. (2008). The clockwork orange Drosophila protein functions as both an activator and a repressor of clock gene expression. J Biol Rhythms, 23(2), 103-116. doi:10.1177/0748730407313817 Robinson, G. E. (1992). Regulation of division of labor in insect societies. Annu Rev Entomol, 37, 637-665. doi:10.1146/annurev.en.37.010192.003225 Robinson, M. D., McCarthy, D. J., & Smyth, G. K. (2010). edgeR: a Bioconductor package for differential expression analysis of digital gene expression data. Bioinformatics, 26(1), 139-140. doi:10.1093/bioinformatics/btp616 Robinson, M. D., & Oshlack, A. (2010). A scaling normalization method for differential expression analysis of RNA-seq data. Genome Biol, 11(3), R25. doi:10.1186/gb-2010- 11-3-r25 Saijo, K., Winner, B., Carson, C. T., Collier, J. G., Boyer, L., Rosenfeld, M. G., . . . Glass, C. K. (2009). A Nurr1/CoREST pathway in microglia and astrocytes protects dopaminergic neurons from inflammation-induced death. Cell, 137(1), 47-59. doi:10.1016/j.cell.2009.01.038

145

Salimullah, M., Sakai, M., Mizuho, S., Plessy, C., & Carninci, P. (2011). NanoCAGE: a high- resolution technique to discover and interrogate cell transcriptomes. Cold Spring Harb Protoc, 2011(1), pdb.prot5559. Sanogo, Y. O., Band, M., Blatti, C., Sinha, S., & Bell, A. M. (2012). Transcriptional regulation of brain gene expression in response to a territorial intrusion. Proc Biol Sci, 279(1749), 4929-4938. doi:10.1098/rspb.2012.2087 Schulz, D. J., & Robinson, G. E. (1999). Biogenic amines and division of labor in honey bee colonies: behaviorally related changes in the antennal lobes and age-related changes in the mushroom bodies. J Comp Physiol A, 184(5), 481-488. Shandala, T., Kortschak, R. D., Gregory, S., & Saint, R. (1999). The Drosophila dead ringer gene is required for early embryonic patterning through regulation of argos and buttonhead expression. Development, 126(19), 4341-4349. Shiraki, T., Kondo, S., Katayama, S., Waki, K., Kasukawa, T., Kawaji, H., . . . Hayashizaki, Y. (2003). Cap analysis gene expression for high-throughput analysis of transcriptional starting point and identification of promoter usage. Proc Natl Acad Sci U S A, 100(26), 15776-15781. doi:10.1073/pnas.2136655100 Sinha, S., Ling, X., Whitfield, C. W., Zhai, C., & Robinson, G. E. (2006). Genome scan for cis- regulatory DNA motifs associated with social behavior in honey bees. Proc Natl Acad Sci U S A, 103(44), 16352-16357. doi:10.1073/pnas.0607448103 Steiger, J. L., Bandyopadhyay, S., Farb, D. H., & Russek, S. J. (2004). cAMP response element-binding protein, activating transcription factor-4, and upstream stimulatory factor differentially control hippocampal GABABR1a and GABABR1b subunit gene expression through alternative promoters. J Neurosci, 24(27), 6115-6126. doi:10.1523/JNEUROSCI.1200-04.2004 Tapscott, S. J. (2005). The circuitry of a master switch: Myod and the regulation of skeletal muscle gene transcription. Development, 132(12), 2685-2695. doi:10.1242/dev.01874 Wang, L., Wang, S., & Li, W. (2012). RSeQC: quality control of RNA-seq experiments. Bioinformatics, 28(16), 2184-2185. doi:10.1093/bioinformatics/bts356 Weasner, B. M., Weasner, B., Deyoung, S. M., Michaels, S. D., & Kumar, J. P. (2009). Transcriptional activities of the Pax6 gene eyeless regulate tissue specificity of ectopic eye formation in Drosophila. Dev Biol, 334(2), 492-502. doi:10.1016/j.ydbio.2009.04.027 Weinstock, G. M., Robinson, G. E., Gibbs, R. A., Worley, K. C., Evans, J. D., Maleszka, R., . . . Honeybee Genome Sequencing, C. (2006). Insights into social insects from the genome of the honeybee Apis mellifera. Nature, 443(7114), 931-949. doi:10.1038/nature05260

146

Whitfield, C. W., Ben-Shahar, Y., Brillet, C., Leoncini, I., Crauser, D., Leconte, Y., . . . Robinson, G. E. (2006). Genomic dissection of behavioral maturation in the honey bee. Proc Natl Acad Sci U S A, 103(44), 16068-16075. doi:10.1073/pnas.0606909103 Whitfield, C. W., Cziko, A. M., & Robinson, G. E. (2003). Gene expression profiles in the brain predict behavior in individual honey bees. Science, 302(5643), 296-299. doi:10.1126/science.1086807 Whitfield, T. W., Wang, J., Collins, P. J., Partridge, E. C., Aldred, S. F., Trinklein, N. D., . . . Weng, Z. (2012). Functional analysis of transcription factor binding sites in human promoters. Genome Biol, 13(9), R50. doi:10.1186/gb-2012-13-9-r50 Zayed, A., & Robinson, G. E. (2012). Understanding the Relationship Between Brain Gene Expression and Social Behavior: Lessons from the Honey Bee. Annu Rev Genet, 46(1), 591-615. doi:doi:10.1146/annurev-genet-110711-155517 Zheng, H., Sun, L., Peng, W., Shen, Y., Wang, Y., Xu, B., . . . Wang, S. (2011). Global identification of transcription start sites in the genome of Apis mellifera using 5'LongSAGE. J Exp Zool B Mol Dev Evol, 316(7), 500-514. doi:10.1002/jez.b.21421

Additional Information CAGEscan Sequences for nurse and forager samples have been deposited in NCBI GEO under the accession number [GSE64315].

147

Chapter 3: Behavioral state and social dynamics influence transcriptional regulatory network plasticity in the honey bee brain*

Abstract Phenotypic plasticity is tightly linked to changes in gene expression, and computational reconstructions of transcriptional regulatory networks (TRNs) can provide comprehensive depictions of the regulatory architecture underlying these changes. A key feature of this architecture is the capacity for the relationships between transcription factors (TFs) and their target genes to be altered by context-specific variables, resulting in novel regulatory relationships that generate behavioral plasticity. However, there is no empirical evidence that links such regulatory network plasticity (“rewiring”) to behavior. We explored this issue in the European honey bee (Apis mellifera), a model of behavioral plasticity with highly related individuals that exhibit stable, long-term differences in behavioral states. Using bioinformatic analyses of the brain transcriptome in conjunction with RNA interference, we show that brain regulatory networks in honey bees are dynamic and modulated by behavioral state, social context and neuroendocrine state. This demonstrates for the first time that behavior and plasticity in brain transcriptomic regulatory architecture are causally related. We also present a hypothetical schema for this relationship by providing evidence that pleiotropic TFs can produce behavior-specific transcriptional responses by activating different downstream TFs, thereby inducing discrete context-dependent transcriptional cascades.

*This work was edited from an in prep manuscript, with headings, figures and tables renumbered.

148

3.1 Introduction A strong reciprocal relationship exists between complex behavior and brain gene expression profiles, with organismal and cellular decision-making processes causally influencing one another (Cardoso, Teles, & Oliveira, 2015; Zayed & Robinson, 2012). However, the regulatory mechanisms connecting behavior and brain gene expression are not well understood. It is known that shifts in transcription factor (TF) expression or activation initiate and maintain alterations in transcriptomic state, leading to behavioral changes through neural plasticity at the neurophysiological and neuroanatomical levels. However, a complementary scenario is that the relationship between TFs and their targets can itself be modulated by plasticity in brain transcriptional regulatory networks (TRNs). This plasticity (or “rewiring”) would allow common TFs to induce context-specific transcriptional programs as a function of the individual’s behavioral and neurophysiological state.

TRN rewiring has been clearly demonstrated at evolutionary timescales, mediated by the addition or removal of transcription factor binding sites (TFBSs) in promoter and enhancer regions (Sorrells & Johnson, 2015; Thompson, Regev, & Roy, 2015). In addition, the relationship between TFs and target genes can be influenced by epigenetic factors that affect the accessibility of TFBSs (Araya et al., 2014; Chen et al., 2013; Roy & Kundu, 2014), as well as changes in the expression of cofactors and partner TFs that influence the affinity of a TF for specific regulatory sequences (Rhee et al., 2014; Stampfel et al., 2015). These forms of rewiring have been observed for developmental processes, for instance, where the establishment of cell identity leads to cell-type specific patterns of TF-target gene expression (Araya et al., 2014; Lorberbaum et al., 2016; Roy & Kundu, 2014). It is now known, however, that many of the processes that induce regulatory network rewiring during development are capable of acting in terminally differentiated cells such as neurons (Sweatt, 2013) at relatively short timescales (Meadows et al., 2016), suggesting that TRN rewiring may also occur in the brain.

We previously modeled a European honey bee (Apis mellifera) brain TRN using a large set of behavioral transcriptomic data bridging multiple experiments and behavioral contexts (Chandrasekaran et al., 2011). The results from this TRN unexpectedly implicated (but did not directly test for) the possibility of network rewiring as a function of behavior. For the present study we therefore use the honey bee to test the hypothesis that regulatory network rewiring is a dynamic process that can rapidly influence the brain transcriptome and behavior.

149

Social insects such as honey bees are excellent models to study the mechanisms underlying transcriptomic and behavioral plasticity. Honey bee colonies exhibit strong patterns of division of labor, based on the behavioral maturation of tens of thousands of closely related non-reproductive individuals (“workers”) that perform dramatically different behaviors on a stable, long-term basis (Robinson, 1992). These stable behavioral states exist as a function of a complex interplay of genotypic, neuroendocrine, developmental, social and abiotic factors during development and adulthood. Adult worker honey bees live for about 6 weeks; they generally specialize on brood care and other within-hive tasks during the first three weeks of adult life and then shift to foraging for the remainder of their life in a process known as behavioral maturation. In addition, bees at a given stage of behavioral maturation can specialize for days on particular tasks, such as nest defense. Individual honey bees can speed up, slow down or reverse transitions between stages of maturation, thus exhibiting both stable behavioral states and strong behavioral plasticity. This makes honey bees particularly useful for this study.

To investigate the connections between behavior and brain regulatory network plasticity, we first used bioinformatic analyses of extensive brain transcriptome data to infer the existence of TRN rewiring in the adult bee brain. We then perturbed candidate TFs identified from these analyses with RNA interference (RNAi) and neuroendocrine treatments. We observed the effects of the perturbations on behavior using field and laboratory assays for behavioral maturation (age at onset of foraging and brood care in the context of queen rearing, respectively), and a laboratory assay for aggression in the context of nest defense. Studying these distinct behavioral contexts allowed us to determine how brain transcriptional regulatory architecture shifts depending on an individual’s behavioral state, social environment and neuroendocrine state, as well as elucidate the temporal aspects of the connection between TRN rewiring and behavioral plasticity.

3.2 Results 3.2.1 Signatures of brain transcriptional regulatory network rewiring inferred from behavior- specific transcriptomic profiles To explore whether TRN rewiring occurs across distinct behavioral states, we drew on previously published brain transcriptomic profiles related to three behavioral contexts: foraging , aggression, and behavioral maturation (Alaux, Sinha, et al., 2009). Working with a previously published model of a honey bee brain TRN (Chandrasekaran et al., 2011), we compared the quantitative relationship between the expression of individual TFs and their predicted target genes across each of these contexts, as a measure of behaviorally related TRN rewiring.

150

We used linear regression for this analysis. While the assumption of a linear relationship between a TF and its target genes is clearly an oversimplification (given that no gene is regulated by a single TF), using linear models previously resulted in the accurate prediction of TF-target gene relationships for about 25% of the genes expressed in the honey bee brain (Chandrasekaran et al., 2011). This suggests that the assumption of linearity also can be used as a heuristic tool to probe TRN rewiring.

We therefore modeled TF-target relationships across behavioral contexts using Analysis of Covariance (ANCOVA), with the TF as a continuous predictor of target gene expression, and the behavioral context as a discrete variable. A significant interaction term (FDR < 0.1) between these two variables indicates a change in the regulatory relationship between the TF and target gene between contexts (see Figure 3.1a for one TF-target gene example). This analysis was performed for 20 TFs predicted to regulate ca. 75% of the genes in the above-mentioned bee brain TRN (Chandrasekaran et al., 2011). Signatures of TRN rewiring in the subnetwork of each TF were inferred by comparing the frequency of significant contrasts across levels of the interaction term between the set of predicted target genes and a background consisting of the remaining probes on the array (using Fisher Exact tests, FDR<0.05). We also determined the effect size for each comparison by comparing the Odds Ratio (OR), calculated by dividing the proportion of target genes with significant TF x Context interaction effects by the proportion of the transcriptomic background that had significant interaction effects.

The putative targets of 14 out of the 20 TFs we studied were overrepresented relative to background genes when testing for a main effect of TF expression (S1 Table). This indicates that our results are largely concordant with the predictions of the bee brain TRN (Chandrasekaran et al., 2011) despite using different methodology and only a subset of the samples that the original TRN was trained on. Although it was common to find a main effect for the TFs on their target genes, signatures of TRN rewiring were found for only four TFs: Broad, Yl-1, Trithorax-like (Trl) and Creb (Figure 3.1b, Table 3.1). Broad and Yl-1 exhibited evidence for rewiring across two pairs of contexts. Broad’s regulatory relationships varied between aggression and foraging (OR = 1.57, FDR = 0.013) as well as aggression and behavioral maturation (OR = 1.56, FDR = 0.012), and Yl-1 relationships varied between foraging and behavioral maturation (OR = 2.12, FDR = 0.015) as well as aggression and behavioral maturation (OR = 3.07, FDR = 0.00003). Trl’s relationships

151 varied between foraging and behavioral maturation (OR = 1.92, FDR = 0.032), and Creb’s regulatory relationships varied between aggression and foraging (OR = 1.54, FDR = 0.025).

3.2.2 RNA interference reduces expression of broad, ftz-1 and their target genes We focused on Broad to experimentally test for a direct connection between network rewiring and behavior, as several factors made it an ideal candidate for such assays. Not only was Broad one of only two TFs to exhibit a signature of plasticity between more than one pair of contexts, but it also is known to be a pleiotropic integrator of diverse endocrine cascades in insects (Gilbert, 2012). For instance, it mediates insulin, juvenile hormone (JH) and vitellogenin (Gilbert, 2012; Hamilton, 2017) signaling, all of which are known to play a causal role in honey bee behavior (Hamilton, 2017; Page, Rueppell, & Amdam, 2012; Pandey & Bloch, 2015; Zayed & Robinson, 2012). Broad also has consistently been predicted to be an important regulator of honey bee behavior in studies employing diverse bioinformatic methods (Seth A. Ament et al., 2012; Chandrasekaran et al., 2011). Moreover, an increase in the number of Broad cis-regulatory elements has previously been linked to the evolution of bee sociality (Kapheim et al., 2015), implying that its regulatory relationships have been rewired over evolutionary timescales. We also targeted a TF that did not exhibit a strong signature of TRN rewiring in the above analysis to contrast with Broad: Fushi tarazu factor 1 (Ftz-F1). Ftz-F1 is a downstream target of Broad in highly conserved endocrine cascades that are critical to insect development (Gilbert, 2012) and, like Broad, is predicted to be a key regulator of several behavioral states in honey bees (Seth A. Ament et al., 2012; Chandrasekaran et al., 2011).

We applied broad and ftz-f1 RNAi to the brain via direct injection. The efficacy of RNAi injections, measured with qPCR on broad and ftz-f1 expression, respectively, varied depending on the type of RNAi and trial (Appendix Figures A.12-A.14). A total of four (20%) of the trials used in this study did not produce a significant knockdown. However, the inclusion of these trials did not change the results of our gene expression and behavioral analyses, pointing to the overall robustness of the RNAi treatments used in this study (see Appendix Figures A.12-A.14 for a more thorough analysis of RNAi treatment efficacy).

We studied the effects of RNAi on behavior and context-dependent changes in the expression of six of each TF’s predicted target genes in the brain as a function of behavioral state, social environment and neuroendocrine state (by manipulating JH). JH plays a strong role in regulating behavioral maturation in honey bees (Robinson, 1992), and it relies on the TFs Broad and Ftz-F1

152 to mediate its transcription-dependent effects on cell physiology (Gilbert, 2012). Based on the position of Broad in the brain TRN (16), we hypothesized that knockdown of Broad expression would result in behavioral changes in behavioral maturation, brood care and aggression and that its relationship with its target genes would vary across behavioral state. Ftz-F1 knockdown, by contrast, was only predicted to influence behavioral maturation and brood care and should not result in TRN rewiring due to behavioral state.

RNAi altered the expression of several putative target genes of Broad and Ftz-F1 (Appendix Figures A.15 and A.16), confirming that these genes are downstream targets of these TFs. Importantly, Broad knockdown also reduced the expression of Ftz-F1 and many of its target genes (Appendix Figure A.15) but not vice versa (Appendix Figure A.16), indicating that the previously reported hierarchical relationship of these two TFs in Drosophila (Gilbert, 2012) is preserved in the adult honey bee brain. This allowed us to test whether a pleiotropic TF involved in behavior (Broad) induces state-specific transcriptional programs via the recruitment of more specialized downstream regulators (e.g., Ftz-F1).

3.2.3 RNA interference and neuroendocrine manipulations alter behavior The bee brain TRN (16) and other cis-regulatory analyses (Seth A. Ament et al., 2012; Chandrasekaran et al., 2011) predicted that Broad regulates behavioral maturation and aggression, whereas Ftz-F1 was predicted to play a role in regulating behavioral maturation but not aggression.

As expected, broad RNAi influenced both behavioral maturation (determined by age at onset of foraging in the field and brood care assays in the lab) and aggression (determined by the response to foreign bees in laboratory intruder assays). broad RNAi caused a significant decrease in foraging onset age (Figure 3.2a) and the intensity of aggression toward intruder bees (Figure 3.2b, Appendix Figure A.17). broad RNAi also caused a significant increase in the intensity of brood care in a laboratory assay in which bees have an opportunity to rear a queen larva (Figure 3.2d, Appendix Figure A.18).

Similarly, ftz-f1 RNAi significantly decreased the age at onset of foraging (Figure 3.2a) and increased the intensity of brood care (Figure 3.2d, Appendix Figure A.18). However, as predicted, ftz-f1 RNAi did not alter overall levels of aggression (Figure 3.2c, Appendix Figure A.17). It is not our intention to suggest that Ftz-F1 plays no role in aggression; indeed it has been found to be

153 differentially expressed in previous aggression-related studies (Alaux, Sinha, et al., 2009; Rittschof et al., 2014). However, when combined with our finding that Broad directly regulates Ftz- F1 expression these results strongly imply that Broad influences aggression via Ftz-F1 independent pathways.

Administration of the JH analog methoprene (JHA) altered the effect of both broad and ftz-f1 RNAi on brood care (Figure 3.2d and 3.2e, Appendix Figure A.18), as expected. JHA partially rescued the increase in brood care behavior observed as result of RNAi treatments.

3.2.4 broad and ftz-f1 RNA interference reveals that brain TRNs dynamically rewire in adult bees To directly test for evidence of TRN rewiring, we used the intruder assay and ANCOVA (as described previously) to determine how the relationship between a TF and its target genes was affected by RNAi, behavioral state (aggression towards an intruder), and the social environment (exposure to the intruder relative to unexposed controls). As expected given the results of our bioinformatic analyses presented above, we found stronger evidence for TRN rewiring for Broad and its targets relative to Ftz-F1 and its targets as a function of social environment. For Broad, 4 out of 6 genes showed significant interaction effects between the RNAi treatment and social context (exposure to an intruder), suggesting that regulatory network rewiring can occur in a short time after exposure to a socially relevant stimulus (Figure 3.3). In contrast, none of Ftz-F1’s target genes exhibited evidence for regulatory plasticity as a function of intruder exposure (Figure 3.3). However, we found evidence for network rewiring as a function of aggressive behavioral state for targets of both TFs: 1 out of 6 target genes tested for Broad and 2 out of 6 for Ftz-F1 (Figure 3.3).

To further explore how the response to a social stimulus influences TRN rewiring for Broad, we performed a second Broad RNAi experiment that measured brain expression of the same six target genes 0, 60 and 120 minutes after exposure to the intruder (gene expression in the previous experiment was measured 60 minutes after intruder exposure). We found that the relationship between Broad and its target genes changed as a function of the amount of time post intruder exposure for 2 out of 6 target genes (Appendix Figure A.19), further reinforcing the idea that its regulatory relationships are labile and can be rapidly altered by social environment.

After determining that exposure to an intruder and the ensuing aggressive behavioral state could induce TRN rewiring, we assayed whether behavioral states and stimuli related to behavioral

154 maturation could do so as well. Age at onset of foraging requires repeated measurements over the course of days, longer than the efficacy of the RNAi treatment. We therefore used the brood care assay mentioned above. Like foraging, brood care also is age-dependent, but unlike foraging it can be assayed using a single discrete event that can be timed precisely with the RNAi treatment.

Again, Broad targets exhibited an overall stronger signature of TRN rewiring. 5 out of 6 genes showed significant interaction effects of RNAi treatment and social context (exposure to a queen larva; Figure 3.4), and 2 out of 6 targets exhibited a significant interaction effect as a function of behavioral state (the ability to perform brood care; Figure 3.4). For Ftz-F1, plasticity was seen only for 1 out of 6 genes as a result of social context, and for none of its targets as a result of behavioral state (Figure 3.4).

Given Broad’s central role as an integrator of endocrine signaling cascades, we tested whether JHA also induced changes in brain gene expression consistent with TRN rewiring. JHA was administered in the diet of a subset of bees prior to RNAi treatment. After RNAi treatment, these bees were then combined with a group fed a control diet, allowing for a factorial design that accounted for behavior, RNAi and JHA effects. The relationship between Broad and two of its target genes was altered as an effect of this targeted endocrine treatment, and Ftz-F1’s relationships to its target genes changed for a single gene (Figure 3.4).

In summary, all six Broad target genes exhibited evidence for TRN rewiring in at least one context (with five demonstrating significant statistical interaction effects in more than one context). Additionally, two of Broad’s targets (GB14024 and unc-79) showed evidence for TRN rewiring in both our bioinformatic and RNAi studies. Evidence for TRN rewiring was weaker for Ftz-F1, as predicted, with three out of six targets demonstrating significant statistical interaction effects. Overall, despite the small number of target genes tested for each TF, the difference in the total number of rewiring events between each TF and its target genes (14 total events for Broad targets, and 4 total events for Ftz-F1 targets) was statistically significant (Fisher Exact Test, p < 0.05, n = 30 possible events).

3.3 Discussion It is well established that regulatory network rewiring can generate plasticity in developmental phenotypes, over both organismal (Araya et al., 2014; Lorberbaum et al., 2016; Roy & Kundu,

155

2014) and evolutionary (Sorrells & Johnson, 2015; Thompson et al., 2015) time scales. Our findings extend the paradigm of TRN rewiring to the brain, and the shorter timescales associated with neural and behavioral plasticity. We demonstrated that the relationships between behaviorally relevant TFs and their target genes depend on whether the TF is functioning in e.g., an aggressive, caregiving or foraging individual. These results provide evidence for dynamical rewiring of transcriptional regulatory networks in the adult bee brain to produce specific patterns of gene expression that vary as a function of behavioral state, neuroendocrine state and the social environment of the individual.

Our bioinformatics analyses provided evidence for TRN rewiring across behavioral contexts, but suggest that the phenomenon may be limited to specific subsets of TFs, as signatures of TRN rewiring were found for only four out of 20 TFs: Broad, Creb, Trithorax-like (Trl) and Yl-1. Orthologs of these four TFs in Drosophila melanogaster are known to be either highly pleiotropic integrators of cell signaling pathways (Broad and Creb) or epigenetic regulators of gene expression (Yl-1 and Trl). Broad is known to be a critical signal integrator in diverse endocrine cascades (Gilbert, 2012), and was one of only four TFs predicted to regulate all three different behavioral contexts (aggression, maturation and foraging) in the bee brain TRN (Chandrasekaran et al., 2011). Creb is essential for the transduction of kinase signaling cascades related to neuronal activation, plasticity and learning and memory (Benito & Barco, 2010), and has been implicated in gating behavioral responses to sensory stimuli (Gehring et al., 2016) and as a critical regulator of foraging behavior in honey bees (Khamis et al., 2015). Yl-1 is a histone acyltransferase (Kusch et al., 2004) and nucleosome remodeler (Weber, Ramachandran, & Henikoff), while Trl (the insect ortholog of GAGA factor (Farkas et al., 1994)) is a chromatin remodeler that inhibits Polycomb Group activity (Ringrose & Paro, 2007). We therefore speculate that TRN rewiring may rely especially on pleiotropic signal integrators, such as Broad and Creb, or epigenetic regulators, such as Yl-1 and Trl, which can initiate and maintain contextually specific transcriptional programs that depend on the internal state of the individual organism.

We confirmed that Broad is a pleiotropic regulator of honey bee behavior by targeting it with RNAi administered to the brain and comparing its impact on behavior to that of a second TF, Ftz-F1. broad RNAi influenced both behavioral maturation and aggression, while ftz-f1 RNAi influenced only behavioral maturation. Both of these results are consistent with the predicted roles of these TFs in honey bee behavior (Chandrasekaran et al., 2011). Likewise, JHA’s partial rescue of the

156

RNAi’s effects on behavioral maturation is consistent with Broad’s role as a mediator of JH signaling (Gilbert, 2012; Hamilton, 2017).

Although the possibility of off-target and deleterious effects always exists when using RNAi, broad and ftz-f1 RNAi’s influence on behavioral can be seen as “gains of function.” Stimulating the early onset of the most physically and cognitively demanding task in the honey bee repertoire (foraging) and an increase in the intensity of another complex behavior (brood care) is evidence that the treated bees were capable of functioning at a normal and healthy level, and suggests that the RNAi treatment had specific effects on behavioral state.

Given that Broad is upstream of Ftz-F1, it could influence behavioral maturation and aggression through pathways that either involve Ftz-F1 or via Ftz-F1-independent pathways. Since both TFs modulated brood care and the age at onset of foraging behavior in a similar fashion, it is likely that Broad’s influence on these behaviors also involves Ftz-F1. Broad’s effect on aggression, however, is likely to be independent of Ftz-F1, since ftz-f1 RNAi did not affect aggressive behavior. These results lead us to speculate that one possible route for a pleiotropic behavioral regulator like Broad to influence multiple behavioral states is via discrete regulatory subnetworks with branching context specific transcriptional cascades. This hypothesis is consistent with the importance of hierarchy in determining a TF’s contribution towards establishing phenotypic plasticity (Bhardwaj, Kim, & Gerstein, 2010).

Indeed, based on orthology to Drosophila, several of Broad’s tested target genes code for regulatory proteins, including two putative TFs (GB11533 and GB11842), a highly conserved component of the polymerase 2 preinitiation complex (TFfIIE휶), and two proteins involved in intracellular signaling cascades: neuron-specific receptor kinase (Nrk), and Toll-7. The fact that these genes exhibited TRN rewiring supports the idea that Broad (and possibly other pleiotropic TFs) function by altering specific signaling pathways in a context dependent manner. Future studies should therefore test this hypothesis, and focus on modeling rewiring across the entire transcriptome to generate dynamic networks that can capture regulatory shifts at a global level (as in (Tian, Gu, & Ma, 2016)).

The fact that two of the four TFs identified in our screen for signatures of TRN rewiring regulate epigenetic modifications and chromatin structure is also striking. Epigenetic factors (including DNA methylation and histone modifications) have been implicated in behavioral plasticity in honey

157 bees and other social species (Herb et al., 2012; Simola et al., 2016), and are also capable of inducing TRN rewiring (Roy & Kundu, 2014). Therefore, their role in mediating regulatory network rewiring in social contexts should be investigated, and future brain TRN models should attempt to incorporate how epigenetic modifications alter TF-target relationships at the global level.

Since the findings presented in this chapter are derived from whole-brain transcriptomes, they represent an aggregate of the individual states of each neuron and glia in the brain. Yet the honey bee brain is highly compartmentalized and consists of numerous specialized regions and neuronal subtypes. Only a subset of these neurons is likely to be activated in a given social context, and it is now known that even neurons of the same subtype and lineage can exhibit strikingly different transcriptomic profiles (Poulin et al., 2016). Integrating information about cell subtype and activation state is thus essential to fully capture the dynamics of brain TRN rewiring and how it relates to behavioral plasticity. Recent advances in single-cell RNA sequencing make this feasible for the first time, and future studies should utilize these breakthroughs to examine TRN rewiring at the level of individual neurons.

In sum, our results demonstrate that plasticity in brain transcriptomic regulatory architecture and behavior are causally related. The genetic regulatory architecture underlying complex behavior thus appears to be more dynamic than previously appreciated, although further study will be necessary to elucidate the functional significance of this plasticity.

3.4 Materials and Methods 3.4.1 Animals All behavioral experiments and RNAi treatments took place at the University of Illinois Bee Research Facility, Urbana, Illinois between the months of June and October in 2013 and 2014. To minimize the effects of genetic variation on behavior and molecular analyses, all experiments used adult worker bees from colonies headed by queens each instrumentally inseminated with semen from a single drone; due to haplodiploidy this results in an average coefficient of relatedness of 0.75. Honeycomb frames containing pupae were removed from colonies one to four days prior to the beginning of each experiment and maintained in a dark incubator at 34◦ Celsius and 50% relative humidity. 0-24 h old adult workers were obtained from these frames, placed into plexiglass cages in groups of 50, and fed 50% sucrose solution (wt/vol) and pollen paste (45% honey/45% pollen/10% water) ad libidum. The bees were kept in these cages in the

158 incubator for four days prior to RNAi treatment, and an additional two days post treatment (except for the bees used in the foraging experiments, see below).

3.4.2 RNAi-nanoparticle complexes Dicer-substrate (Integrated DNA Technologies, Coralville, Iowa) RNAi constructs (D-siRNA) were designed against exons shared between all predicted isoforms of broad complex (br) or fushi- tarazu transcription factor 1 (ftz-f1). A cocktail of three such constructs (Appendix Table B.9) in equivalent concentrations was used to target each gene. Perfluorocarbon nanoparticles were fabricated as previously described (Kaneda et al. 2010). To create nanoparticle-D-siRNA complexes, 1uM of the D-siRNA cocktail or an exogenous control (targeted to green fluorescent protein (gfp), Integrated DNA Technologies, Coralville, Iowa) and 2 nM nanoparticles were combined in 1x Phosphate Buffered Saline (Sigma-Aldrich, St. Louis, Missouri). This solution was incubated at room temperature for 30 min with light agitation before use. These nanoparticles have been used previously to administer siRNA to honey bees (Li-Byarlay et al., 2013).

In 2013, 20% high molecular weight dextran (Sigma-Aldrich, St. Louis, Missouri) was added to the nanoparticle complexes post-incubation to increase the viscosity of the solution and prevent it from diffusing away from the head capsule during injection. However, dextran use resulted in a substantial higher mortality, so this practice was discontinued in 2014.

3.4.3 Injections Bees were cold-anesthetized and mounted on a Plasticine (Flair Leisure Products Plc, Cheam, United Kingdom) pedestal; dental floss was used to stabilize the head. Pedestals were placed in a tray of dry ice to ensure that the bees did not wake prior to the end of the injection. Prior to injection, two paint marks (Testors, Rockford, IL, USA) were applied to the individual’s thorax and a preliminary incision was made to the posterior part of the median ocellus using a 28 gauge needle. A 34 gauge Nanofil (World Precision Instruments, Sarasota, Florida) needle was inserted 500 μm into the incision, and 500nL of nanoparticle complexes were administered using a UMP3 Microinjector run by a Micro4 controller (World Precision Instruments, Sarasota, Florida) at a rate of 5 nL/sec. Bees that exhibited backflow of the injection mixture were discarded. After injection, bees were kept anesthetized for ca. 10 min before being returned to their cages to prevent the injected solution from circulating away from the brain. Mortality as a result of the injections was ca. 15-20%, with no statistically significant differences as a function of treatment type (Kruskal- Wallis test, Appendix Figure A.20).

159

3.4.4 Juvenile hormone analog administration In 2014, bees in half of the cages were administered the juvenile hormone analog (JHA) methoprene using a method adapted from established protocols (S. A. Ament et al., 2012; Whitfield et al., 2006). Methoprene was mixed with the pollen paste (20mg/g) until the day prior to RNAi treatment, at which point it was replaced with standard pollen paste. After RNAi treatment, methoprene-treated bees and control diet bees were combined into the same cage and co- housed for the remainder of the experiment. Previous research (S. A. Ament et al., 2012; Whitfield et al., 2006) has shown that this dose is sufficient to cause precocious foraging and massive changes in brain gene expression.

3.4.5 Behavioral maturation: Precocious foraging Single-cohort colonies initially comprised of ca. 2,000 one-day-old worker bees were established according to previous protocols (Robinson, Page, Strambi, & Strambi, 1989). Each colony was given two honeycomb frames of honey, one frame of pollen, one empty frame and a mated queen unrelated to the worker bees. Each colony was formed two days prior to the eclosion of the treated bees. Instead of being housed in an incubator, plexiglass cages containing these bees (provisioned as noted previously and given a piece of comb from the parent colony) were housed within the colony prior to RNAi treatment. Immediately after RNAi treatment, treated and control bees were each tagged with a pair of RFID (p-chip) transponders (Pharmaseq, Princeton, New Jersey) (Tenczar, Lutz, Rao, Goldenfeld, & Robinson, 2014). The bees were then returned to their cages and placed within the colony until their release 12 h later. Together with the presence of common pieces of comb, the co-housing procedure ensured that the other bees in the colony were not aggressive toward the treated bees. Up to three cohorts of bees were added to each colony on consecutive days. The flight activity of the treated and control bees was continuously by a pair of barcode readers (Pharmaseq, Princeton, New Jersey) mounted at the hive entrance that monitored all outgoing and incoming trips (Tenczar et al., 2014). Data were collected continuously for 10 days post-RNAi treatment, and age at onset of foraging information was extracted as described in (Tenczar et al., 2014), summarized below.

3.4.6 Aggression assays Two days after RNAi treatment, bees were transferred to petri dishes provisioned with honey, water, a pollen ball and a piece of wax comb from their original cage. In 2013, groups of six (3 control RNAi and 3 treatment RNAi) were used; in 2014 groups of eight were used , consisting of

160 two bees from each JHA x RNAi treatment combination, yielding a 2x2 factorial. Each group member was marked uniquely to enable behavioral analyses at the individual level. The bees were then placed in a climate controlled testing room (28◦ Celsius) with normal fluorescent lighting and allowed to acclimate for at least 60 minutes prior to testing.

Aggression assays were adapted from a previously developed intruder assay (Li-Byarlay, Rittschof, Massey, Pittendrigh, & Robinson, 2014). An intruder collected from the entrance of an unrelated colony immediately prior to the assay was introduced to the group of bees. The bees were then observed for 5 min, after which the intruder was removed from the cage, regardless of whether it was still alive. The bees were then exposed to 5uL of isopentyl acetate (Sigma-Aldrich, St. Louis, Missouri), the active ingredient in honey bee alarm pheromone, diluted 1:10 with mineral oil as described previously (Alaux & Robinson, 2007). This additional treatment was added to simulate a high level threat to the group, even if none of the residents responded to the intruder in an aggressive manner. Previous studies have shown that this intruder assay produces behavior that is consistent with aggressive defense behavior in colonies in the field [37].

In 2013, treatment and control bees were collected one hour after the presentation of the stimulus by flash-freezing them in liquid nitrogen. Unexposed control groups served as a baseline for gene expression assays. These groups were acclimated to the same chamber and collected during the same time frame, but were not administered an intruder, IPA or control stimulus. In 2014, bees were instead collected either immediately after the end of the assay (before the assay could elicit changes in transcription) or 60-120 minutes later. Those groups collected immediately after the end of the assay served as the baseline for all gene expression studies.

3.4.7 Behavioral maturation: Brood care assays Brood care assays were performed as described previously (Shpigler & Robinson, 2015). A waxen naturally built cell containing a 3- or 4-day old queen larva was introduced to a cage of bees, and the interactions between the residents and the queen were scan-sampled at 10 sec intervals. The queen cell was removed after 10 min of observations. Control cages were each given an empty queen cell to determine whether the RNAi treatment caused any differences in general sensitivity to stimuli. Previous studies have shown that this assay produces behavior that is consistent with brood care behavior in colonies in the field [39]. Collections followed the same protocol as the aggression assays performed in 2014, described above.

161

3.4.8 Sample preparation and quantitative PCR Sample preparation and qPCR were performed using established protocols (Ament et al., 2011). Whole brains were dissected into RNA-later ICE (Thermo Fisher Scientific, Waltham, Massachusetts) at -80◦ Celsius, homogenized and extracted using RNeasy spin columns (Qiagen, Valencia, California) with DNase treatment (Invitrogen, Carlsbad, California). RNA was quantified using a Nanodrop spectrometer (Thermo Fisher Scientific, Waltham, Massachusetts) and Qubit fluorimeter (Thermo Fisher Scientific, Waltham, Massachusetts), and cDNA syntheses were performed using Arrayscript (Thermo Fisher Scientific, Waltham, Massachusetts) reverse transcriptase. An exogenous RNA spike-in (Root Cap Protein 1 from Arabidopsis thaliana) was used to assess cDNA synthesis efficiency. Quantitative PCR (qPCR) was performed using SYBR Green dye (Thermo Fisher Scientific, Waltham, Massachusetts) on an ABI Prism 7900ht (for samples analyzed in 2013) or ABI QuantStudio 6 (for samples analyzed in 2014). Sample-probe combinations with an inter-triplicate coefficient of variation (CV) higher than 30% were discarded. For qPCR probes see Appendix Table B.9.

3.4.9 ANCOVA analysis for signatures of brain TRN plasticity Our bioinformatic analyses were conducted on transcriptomic results from previously published studies (Alaux, Le Conte, et al., 2009; Alaux, Sinha, et al., 2009) conducted with custom honey bee microarrays (Array Express Accession Numbers E-TABM-604, E-TABM-605, E-TABM-606, and E-TABM-607). These experiments aimed at assessing the interaction between genotype and environmental factors using cross-fostered bees in three different behavioral contexts: aggression (soldier, guard and alarm pheromone exposed bees), foraging, and behavioral maturation (four day old and 14 day old bees). Although the experiments in question were not designed to assess transcriptomic differences between foraging, behavioral maturation and aggression, they were performed by the same individual, used bees of the same genotype and colony of origin, and were performed during the same three-month period. Still, to further insure that the contribution of these factors was minimized we processed mean-centered expression values from the microarrays using RUVcorr (Freytag, Gagnon-Bartsch, Speed, & Bahlo, 2015), an implementation of the Remove Unwanted Variation (RUV) algorithm (Gagnon-Bartsch & Speed, 2012) that is designed for use in gene co-expression analyses (S1 File). This algorithm uses internal reference genes that are unlikely to vary across experimental conditions to correct for technical variability in the microarray data. In our case, we used a set of 2,000 probes identified as having the lowest variability across all samples (excluding the genes previously identified as putative targets for the TFs in question). Ridge Regression (Freytag et al., 2015) was then performed to adjust the

162 expression of the remaining genes (Appendix Figure A.21), and these values were then used in an Analysis of Covariance (ANCOVA; Appendix Dataset C.10).

The ANCOVA was performed using the Limma package (Ritchie et al., 2015) in R. The expression values of TFs of interest were used as continuous covariates, and the behavioral context was modeled as a categorical variable with three levels (for aggression, foraging and behavioral maturation). The interaction term of these two variables therefore assesses the degree of change in the relationship between the TF and target gene expression across the behavioral contexts, and contrasts between the levels of the interaction term can be used to determine the specific contexts responsible for this change. We performed these analyses on 20 TFs (Table 3.1) that were previously identified as putative regulators of the bee brain transcriptome (Chandrasekaran et al., 2011), relating their expression (S2 File) to a curated set of probes (9,544 in total; S2 and S3 Files) from the arrays used in (Chandrasekaran et al., 2011). Signatures of network rewiring between each pair of contexts were then inferred by comparing the total number of significant (FDR < 0.10) contrasts between a TF’s putative targets (identified in (Chandrasekaran et al., 2011); S3 File) and the remainder of the probes (one-sided Fisher Exact Test, FDR < 0.05). Only if at least 10% of a TF’s target genes exhibited a significant contrast between the relevant pair of behavioral contexts was it considered for this analysis. Two TFs (Broad and NF-κB) were represented by multiple probes on the array. To account for this, the protocol was run independently for each individual TF probe, and significant effects were compiled across the target gene and background sets for the Fisher Exact test.

3.4.10 Observation and analysis of age at onset of foraging The age at onset of foraging was determined with an algorithm that relates the proportion of p- chip reads during peak foraging hours to the proportion during likely orientation times (Tenczar et al., 2014). Bees with more than eight reads for at least two consecutive days, with greater than half of their reads collected during peak foraging hours (1200-1500 CST) were classified as foragers (S4 File). A previous study using similar, but less stringent, thresholds found an excellent correspondence between this metric and manual observations (Tenczar et al., 2014). Up to three cohorts of bees were treated with RNAi and/or JHA treatment on consecutive days and added to each background colony. The age at onset of foraging data were analyzed with a Cox Proportional Hazards (coxph in R; S1 File) model (Ben-Shahar, Robichon, Sokolowski, & Robinson, 2002), stratified by the colony the bees were added to and the cohort they belonged to within this colony.

163

Bees that died before the end of the experiment and did not become foragers were removed from the analysis, since the time of their death could not be accurately determined.

3.4.11 Observation and analysis of aggression and brood care behavior Behavioral data from the aggression assay were scored using an index adapted from (Li-Byarlay et al., 2013). Bees were assayed using a scan sampling approach for 5 min, with a blinded observer noting all interactions between residents and the intruder during 10 sec intervals (Table 3.2). These values were summed for each 10 sec window that an individual resident interacted with the intruder, and the total score associated with each bee was normalized by the duration of the potential interaction time (i.e., if a resident or intruder died prior to the end of the assay, additional bins would not be factored into the normalized score) (S4 File). The average index for each treatment group of bees was used as the experimental unit, with individual bees regarded as subsamples. Because of the invasive nature of the RNAi treatments and the potential for treatment-related injuries to interfere with normal behavior, individuals that did not respond to the intruder (30% across all assays, with no significant differences between treatments) were not scored and were discarded from the study. Brood care behavior was measured similarly and scored using an index adapted from (Shpigler & Robinson, 2015) (Table 3.3; Appendix Dataset C.13).

The distribution of aggression and brood care scores across samples deviated significantly from normality, so a Box-Cox transformation was applied as a correction. Post transformation, all data from 2013 met requirements of normality and homoscedasticity. Behavioral data from 2014 met all assumptions post transformation, except for skewness and, in some instances, kurtosis(likely due to the lower number of pseudoreplicates in each sample); however, mixed models have been shown to be robust to violations of skewness and kurtosis when total sample sizes larger than 45 are used (Arnau, Bendayan, Blanca, & Bono, 2013). Therefore, we analyzed the transformed data using a linear mixed effects model (the lme4 package in R (Bates, Mächler, Bolker, & Walker, 2015) with Type III Sums of Squares and the Kenward-Roger approximation of degrees of freedom; S1 File). To account for group-wise differences in variables such as the stimulus presented (which cannot be easily quantified or controlled for across groups), we included a random variable that accounted for the cage that each set of bees belonged to (nested within the date of testing and the bees’ colony of origin). We also initially blocked for the experimental observer and included the time of testing as a linear covariate. However, the influence of observer

164 and the time of testing were not found to be significant, so these variables were removed from the final model.

The effects of the RNAi (br and ftz-f1 RNAi) were analyzed independently, since the injections were performed on distinct sets of bees. In 2014, JHA treatment was added as a response variable of interest, and the interaction between JHA and br/ftz-f1 RNAi was modeled as well.

3.4.12 ANCOVA analyses of qPCR data Expression values for each gene were normalized to the geometric mean of four reference genes (S8, RP49, GAPDH, and eIF1-a). Each reference gene was assessed individually to ensure that the coefficient of variation (CV) was less than 40% in each experiment/colony combination, and there were no significant differences in reference gene expression between experimental groups (S5 File).

The influence of the TFs on candidate target genes was determined in two ways: 1) using a multifactorial ANOVA and looking at the effect of RNAi on target gene expression or 2) modeling TF expression levels as a continuous variable and using an ANCOVA to determine the relationship between TF and target expression. In both cases, a fixed effects model was used with Type III Sums of Squares (S1 File). The samples’ colony of origin was included as a blocking factor; the time and date of testing were tested but did not have a significant effect on gene expression and were dropped from the final models.

Context-dependent effects of behavioral state (how the bee responded to the social stimulus), social environment (i.e., the presence of an intruder or queen cell), and endocrine state (whether the bees were treated with JHA or not) were determined by examining the two-way interactions of these response variables with br expression (via ANCOVA).

During the 2013 field season, the baseline for social context was provided by matched control groups that were not exposed to an intruder or alarm pheromone, but were otherwise treated in the same manner. The effect of behavioral state on transcription was therefore nested within stimulus exposure (since behavioral state could not be ascertained for the unexposed group). In 2014, by contrast, the data were organized in a time series that was analyzed as a split-plot in time with three levels (see above), allowing for the direct comparison of (non)responding individuals before gene and after expression changes were elicited by the social stimulus.

165

Therefore, both behavioral state and the response to stimulus could be resolved allowing each to be treated as a main effect, with their interaction term delineating the difference between the way (non)responders were influenced by the social stimulus. Additionally, the effect of methoprene on gene expression and the TFs’ regulatory relationships was also assessed.

In the behavioral maturation experiment, the efficacy of RNAi treatments was confirmed by sampling treated bees on the day they were released into the colony. The influence of RNAi was analyzed as above, but blocking only for colony of origin.

To ensure that broad knockdowns did not induce global changes in transcription, we also assessed the expression of a panel of six putative targets of a third TF (CG17912) that were not predicted targets of Broad and Ftz-F1. The fact that only one of these genes was differentially expressed in response to RNAi indicates that the effect of these treatments was largely specific to the putative targets (Appendix Figure A.22; Appendix Dataset C.14).

Acknowledgements I would like to thank my coauthors Ian M. Traniello, Allyson M. Ray, Arminius S. Caldwell, Samuel A. Wickline, and Gene E. Robinson for their contributions to this work. I would like to acknowledge Daniel C. Nye for assistance with honey bee rearing and colony maintenance in the field, Emma E. Murdoch for assistance with data collection, Michael J. Scott for synthesizing the perfluorocarbon nanoparticles, and Tom Newman and Amy Cash Ahmed for

166 their help and advice with molecular biology. Finally, I also thank Claudia C. Lutz and the members of the Robinson lab for reviewing the manuscript and offering critical feedback.

167

3.5 Figures

Figure 3.1. ANCOVA analyses of transcriptomic data reveal signatures of regulatory network plasticity as a function of behavioral state. (A) Shifts in the relationship between a TF and a representative target gene across behavioral states can be represented by linear regression. Shaded areas represent the 95% confidence intervals. (B) Proportion of putative target genes that lack evidence of TRN rewiring (the Main Effect only column) or that exhibit TRN rewiring for a TF with signatures of plasticity (Broad) and one without (Ftz-F1). (C) Proportion of putative target genes exhibiting TRN rewiring for four TFs that exhibit signatures of plasticity. Data from a total of 3 experiments were used for this analysis with the number of individual brain transcriptomic profiles as follows: behavioral maturation (n=39), aggression (n=118) and foraging (n=40).

168

Figure 3.2. Effect of RNAi on bee behavior. (A) broad and ftz-f1 RNAi both decreased the age at onset of foraging behavior (Cox Proportional Hazards). broad RNAi decreased aggression, but ftz-f1 RNAi had no effect on aggression (B&C), while both broad and ftz-f1 RNAi increased brood care (D&E). Different letters indicate statistically different groups (linear mixed model with Tukey’s post-hoc correction, p < 0.05). In all box plots, center lines show the group median and pluses indicate the group mean; the limits represent the 25th and 75th percentiles, and whiskers extend 1.5 times the interquartile range from these percentiles. For information on colony variability for these measures, see S1 and S2 Figs.

169

Figure 3.3. ANCOVA analyses of qPCR data reveal regulatory network plasticity related to aggressive behavior and social context. A) Each node depicts one Broad or Ftz-F1 target gene, and each edge represents a significant main effect or an interaction effect that is indicative of rewiring (p < 0.05). B) Each column represents the proportion of Broad (blue) and Ftz-F1 (red) target genes that undergo rewiring in a particular context. “Aggressive State” refers to bees that responded aggressively to an intruder, while “Intruder Exposure” refers to the social context (i.e., either whether the bee was present in an assay that received either an intruder or no stimulus).

170

Figure 3.4. ANCOVA analyses of qPCR data reveal regulatory network plasticity related to brood care behavior, the amount of time after exposure to a social context, and neuroendocrine state. A) Each node depicts one of the Broad or Ftz-F1 target gene, and each edge represents a significant main effect or an interaction effect that’s indicative of rewiring (p<0.05). B) Each column represents the proportion of Broad (blue) and Ftz-F1 (red) target genes that undergo rewiring in a particular context. “Brood Care State” refers to bees that responded to the presence of a queen larva by feeding it; “Time Post-Brood Care Stimulus” refers to the response to the social context (i.e., 0, 60, and 120 minutes post queen larva exposure); “JHA” refers to whether the bees were treated with JHA or not.

171

3.6 Tables Table 3.1. Analysis of Covariance across multiple behavioral states reveals signatures of TRN rewiring.

172

Table 3.2. Aggression Index

173

Table 3.3: Brood Care Index

174

References Alaux, C., Le Conte, Y., Adams, H. A., Rodriguez-Zas, S., Grozinger, C. M., Sinha, S., & Robinson, G. E. (2009). Regulation of brain gene expression in honey bees by brood pheromone. Genes Brain and Behavior, 8(3), 309-319. doi:10.1111/j.1601- 183X.2009.00480.x Alaux, C., & Robinson, G. E. (2007). Alarm pheromone induces immediate-early gene expression and slow behavioral response in honey bees. Journal of Chemical Ecology, 33(7), 1346- 1350. doi:10.1007/s10886-007-9301-6 Alaux, C., Sinha, S., Hasadsri, L., Hunt, G. J., Guzman-Novoa, E., DeGrandi-Hoffman, G., . . . Robinson, G. E. (2009). Honey bee aggression supports a link between gene regulation and behavioral evolution. Proceedings of the National Academy of Sciences of the United States of America, 106(36), 15400-15405. doi:10.1073/pnas.0907043106 Ament, S. A., Blatti, C. A., Alaux, C., Wheeler, M. M., Toth, A. L., Le Conte, Y., . . . Sinha, S. (2012). New meta-analysis tools reveal common transcriptional regulatory basis for multiple determinants of behavior. Proceedings of the National Academy of Sciences of the United States of America, 109(26), E1801-E1810. doi:10.1073/pnas.1205283109 Ament, S. A., Chan, Q. W., Wheeler, M. M., Nixon, S. E., Johnson, S. P., Rodriguez-Zas, S. L., . . . Robinson, G. E. (2011). Mechanisms of stable lipid loss in a social insect. Journal of Experimental Biology, 214(22), 3808-3821. doi:10.1242/jeb.060244 Ament, S. A., Wang, Y., Chen, C. C., Blatti, C. A., Hong, F., Liang, Z. Z. S., . . . Robinson, G. E. (2012). The transcription factor Ultraspiracle influences honey bee social behavior and behavior-related gene expression. Plos Genetics, 8(3), 15. doi:10.1371/journal.pgen.1002596 Araya, C. L., Kawli, T., Kundaje, A., Jiang, L. X., Wu, B. J., Vafeados, D., . . . Snyder, M. (2014). Regulatory analysis of the C. elegans genome with spatiotemporal resolution. Nature, 512(7515), 400-U363. doi:10.1038/nature13497 Arnau, J., Bendayan, R., Blanca, M. J., & Bono, R. (2013). The effect of skewness and kurtosis on the robustness of linear mixed models. Behavior Research Methods, 45(3), 873-879. doi:10.3758/s13428-012-0306-x Bates, D., Mächler, M., Bolker, B., & Walker, S. (2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software; Vol 1, Issue 1 (2015). Ben-Shahar, Y., Robichon, A., Sokolowski, M. B., & Robinson, G. E. (2002). Influence of gene action across different time scales on behavior. Science, 296(5568), 741-744. doi:10.1126/science.1069911

175

Benito, E., & Barco, A. (2010). CREB's control of intrinsic and synaptic plasticity: implications for CREB-dependent memory models. Trends in Neurosciences, 33(5), 230-240. doi:10.1016/j.tins.2010.02.001 Bhardwaj, N., Kim, P. M., & Gerstein, M. B. (2010). Rewiring of transcriptional regulatory networks: Hierarchy, rather than connectivity, better reflects the importance of regulators. Science Signaling, 3(146). doi:10.1126/scisignal.2001014 Cardoso, S. D., Teles, M. C., & Oliveira, R. F. (2015). Neurogenomic mechanisms of social plasticity. Journal of Experimental Biology, 218(1), 140-149. Chandrasekaran, S., Ament, S. A., Eddy, J. A., Rodriguez-Zas, S. L., Schatz, B. R., Price, N. D., & Robinson, G. E. (2011). Behavior-specific changes in transcriptional modules lead to distinct and predictable neurogenomic states. Proceedings of the National Academy of Sciences of the United States of America, 108(44), 18020-18025. doi:10.1073/pnas.1114093108 Chen, C. C., Xiao, S., Xie, D., Cao, X., Song, C. X., Wang, T., . . . Zhong, S. (2013). Understanding variation in transcription factor binding by modeling transcription factor genome- epigenome interactions. Plos Computational Biology, 9(12). doi:10.1371/journal.pcbi.1003367 Farkas, G., Gausz, J., Galloni, M., Reuter, G., Gyurkovics, H., & Karch, F. (1994). The Trithorax- like gene encodes the Drosophila GAGA factor. Nature, 371, 806. doi:10.1038/371806a0 Freytag, S., Gagnon-Bartsch, J., Speed, T. P., & Bahlo, M. (2015). Systematic noise degrades gene co-expression signals but can be corrected. Bmc Bioinformatics, 16. doi:10.1186/s12859-015-0745-3 Gagnon-Bartsch, J. A., & Speed, T. P. (2012). Using control genes to correct for unwanted variation in microarray data. Biostatistics, 13(3), 539-552. doi:10.1093/biostatistics/kxr034 Gehring, K. B., Heufelder, K., Feige, J., Bauer, P., Dyck, Y., Ehrhardt, L., . . . Eisenhardt, D. (2016). Involvement of phosphorylated Apis mellifera CREB in gating a honeybee's behavioral response to an external stimulus. Learning & Memory, 23(5), 195-207. doi:10.1101/lm.040964.115 Gilbert, L. I. (2012). Insect endocrinology. In (pp. 1 online resource (x, 577 p.)). Retrieved from ScienceDirect - Full text online http://www.library.uiuc.edu/proxy/go.php?url=http://www.sciencedirect.com/science/book /9780123847492 Hamilton, A. R., Shpigler, H., Bloch, G., Wheeler, D.E., Robinson, G.E. . (2017). Endocrine inﬂuences on insect societies. In D. W. a. J. Pfaff, M. (Ed.), Hormones, Brain, and Behavior

176

3rd edition (Vol. Vol 2, pp. 421–451): Oxford: Academic Press. Herb, B. R., Wolschin, F., Hansen, K. D., Aryee, M. J., Langmead, B., Irizarry, R., . . . Feinberg, A. P. (2012). Reversible switching between epigenetic states in honeybee behavioral subcastes. Nature Neuroscience, 15(10), 1371-1373. doi:10.1038/nn.3218 Kapheim, K. M., Pan, H., Li, C., Salzberg, S. L., Puiu, D., Magoc, T., . . . Zhang, G. (2015). Genomic signatures of evolutionary transitions from solitary to group living. Science, 348(6239), 1139-1143. doi:10.1126/science.aaa4788 Khamis, A. M., Hamilton, A. R., Medvedeva, Y. A., Alam, T., Alam, I., Essack, M., . . . Bajic, V. B. (2015). Insights into the transcriptional architecture of behavioral plasticity in the honey bee Apis mellifera. Scientific Reports, 5. doi:10.1038/srep11136 Kusch, T., Florens, L., MacDonald, W. H., Swanson, S. K., Glaser, R. L., Yates, J. R., . . . Workman, J. L. (2004). Acetylation by Tip60 is required for selective histone variant exchange at DNA lesions. Science, 306(5704), 2084-2087. doi:10.1126/science.1103455 Li-Byarlay, H., Li, Y., Stroud, H., Feng, S. H., Newman, T. C., Kaneda, M., . . . Robinson, G. E. (2013). RNA interference knockdown of DNA methyltransferase 3 affects gene alternative splicing in the honey bee. Proceedings of the National Academy of Sciences of the United States of America, 110(31), 12750-12755. doi:10.1073/pnas.1310735110 Li-Byarlay, H., Rittschof, C. C., Massey, J. H., Pittendrigh, B. R., & Robinson, G. E. (2014). Socially responsive effects of brain oxidative metabolism on aggression. Proceedings of the National Academy of Sciences of the United States of America, 111(34), 12533-12537. doi:10.1073/pnas.1412306111 Lorberbaum, D. S., Ramos, A. I., Peterson, K. A., Carpenter, B. S., Parker, D. S., De, S., . . . Barolo, S. (2016). An ancient yet flexible cis-regulatory architecture allows localized Hedgehog tuning by patched/Ptch1. Elife, 5. doi:10.7554/eLife.13550 Meadows, J. P., Guzman-Karlsson, M. C., Phillips, S., Brown, J. A., Strange, S. K., Sweatt, J. D., & Hablitz, J. J. (2016). Dynamic DNA methylation regulates neuronal intrinsic membrane excitability. Science Signaling, 9(442), 14. doi:10.1126/scisignal.aaf5642 Page, R. E., Rueppell, O., & Amdam, G. V. (2012). Genetics of reproduction and regulation of honeybee (Apis mellifera L.) social behavior. In B. L. Bassler (Ed.), Annual Review of Genetics, Vol 46 (Vol. 46, pp. 97-119). Palo Alto: Annual Reviews. Pandey, A., & Bloch, G. (2015). Juvenile hormone and ecdysteroids as major regulators of brain and behavior in bees. Current Opinion in Insect Science, 12, 26-37. doi:10.1016/j.cois.2015.09.006 Poulin, J. F., Tasic, B., Hjerling-Leffler, J., Trimarchi, J. M., & Awatramani, R. (2016).

177

Disentangling neural cell diversity using single-cell transcriptomics. Nature Neuroscience, 19(9), 1131-1141. doi:10.1038/nn.4366 Rhee, D. Y., Cho, D. Y., Zhai, B., Slattery, M., Ma, L. J., Mintseris, J., . . . Artavanis-Tsakonas, S. (2014). Transcription factor networks in Drosophila melanogaster. Cell Reports, 8(6), 2031-2043. doi:10.1016/j.celrep.2014.08.038 Ringrose, L., & Paro, R. (2007). Polycomb/Trithorax response elements and epigenetic memory of cell identity. Development, 134(2), 223-232. doi:10.1242/dev.02723 Ritchie, M. E., Phipson, B., Wu, D., Hu, Y. F., Law, C. W., Shi, W., & Smyth, G. K. (2015). Limma powers differential expression analyses for RNA-sequencing and microarray studies. Nucleic Acids Research, 43(7). doi:10.1093/nar/gkv007 Rittschof, C. C., Bukhari, S. A., Sloofman, L. G., Troy, J. M., Caetano-Anolles, D., Cash-Ahmed, A., . . . Stubbs, L. (2014). Neuromolecular responses to social challenge: common mechanisms across mouse, stickleback fish, and honey bee. Proceedings of the National Academy of Sciences of the United States of America, 111(50), 17929-17934. doi:10.1073/pnas.1420369111 Robinson, G. E. (1992). Regulation of division-of-labor in insect societies. Annual Review of Entomology, 37, 637-665. doi:10.1146/annurev.en.37.010192.003225 Robinson, G. E., Page, R. E., Strambi, C., & Strambi, A. (1989). Hormonal and genetic control of behavioral integration in honey bee colonies. Science, 246(4926), 109-111. doi:10.1126/science.246.4926.109 Roy, S., & Kundu, T. K. (2014). Gene regulatory networks and epigenetic modifications in cell differentiation. Iubmb Life, 66(2), 100-109. doi:10.1002/iub.1249 Shpigler, H. Y., & Robinson, G. E. (2015). Laboratory assay of brood care for quantitative analyses of individual differences in honey bee (Apis mellifera) affiliative behavior. Plos One, 10(11). doi:10.1371/journal.pone.0143183 Simola, D. F., Graham, R. J., Brady, C. M., Enzmann, B. L., Desplan, C., Ray, A., . . . Berger, S. L. (2016). Epigenetic (re)programming of caste-specific behavior in the ant Camponotus floridanus. Science, 351(6268), 42-NIL_69. Sorrells, T. R., & Johnson, A. D. (2015). Making sense of transcription networks. Cell, 161(4), 714-723. doi:10.1016/j.cell.2015.04.014 Stampfel, G., Kazmar, T., Frank, O., Wienerroither, S., Reiter, F., & Stark, A. (2015). Transcriptional regulators form diverse groups with context-dependent regulatory functions. Nature, 528(7580), 147-+. doi:10.1038/nature15545 Sweatt, J. D. (2013). The emerging field of neuroepigenetics. Neuron, 80(3), 624-632.

178

doi:10.1016/j.neuron.2013.10.023 Tenczar, P., Lutz, C. C., Rao, V. D., Goldenfeld, N., & Robinson, G. E. (2014). Automated monitoring reveals extreme interindividual variation and plasticity in honeybee foraging activity levels. Animal Behaviour, 95, 41-48. doi:10.1016/j.anbehav.2014.06.006 Thompson, D., Regev, A., & Roy, S. (2015). Comparative analysis of gene regulatory networks: From network reconstruction to evolution. In R. Schekman (Ed.), Annual Review of Cell and Developmental Biology, Vol 31 (Vol. 31, pp. 399-428). Tian, D. C., Gu, Q. Q., & Ma, J. (2016). Identifying gene regulatory network rewiring using latent differential graphical models. Nucleic Acids Research, 44(17). doi:10.1093/nar/gkw581 Weber, Christopher M., Ramachandran, S., & Henikoff, S. Nucleosomes are context-specific, H2A.Z-modulated barriers to RNA polymerase. Molecular Cell, 53(5), 819-830. doi:10.1016/j.molcel.2014.02.014 Whitfield, C. W., Ben-Shahar, Y., Brillet, C., Leoncini, I., Crauser, D., Le Conte, Y., . . . Robinson, G. E. (2006). Genomic dissection of behavioral maturation in the honey bee. Proceedings of the National Academy of Sciences of the United States of America, 103(44), 16068- 16075. doi:10.1073/pnas.0606909103 Zayed, A., & Robinson, G. E. (2012). Understanding the relationship between brain gene expression and social behavior: Lessons from the honey bee. Annual Review of Genetics, Vol 46, 46, 591-615. doi:10.1146/annurev-genet-110711-155517

179

Chapter 4: Investigating the link between social networks and division of labor

Abstract Complex societies are often characterized by a division of labor that can be adjusted to maximize productivity in response to changes in the individual, society and environment. Communication can facilitate the genesis and adaptation of division of labor by coordinating behavioral changes among individual society members. To explore the relationship between division of labor and communication in honey bees, we generated trophallaxis (a form of communication that involves liquid food exchange) social networks using groups of 60 bees. We then performed a series of behavioral assays to determine whether individuals exhibiting socially relevant task specializations such as broodcare, colony defense, wax building, fanning, and vibration signaling, or that were socially unresponsive in the assays, differed in patterns of trophallaxis. We discovered that the tasks that require direct interaction with nestmates (such as colony defense, broodcare and vibration signaling) were associated with a higher number of trophallactic interactions, interaction partners, and an increase in the duration of interactions, whereas social unresponsiveness was associated with fewer trophallactic interactions and interaction partners. We also administered a juvenile hormone analog (methoprene) to a subset of the bees in each group to shift division of labor and determine whether this would perturb the trophallaxis social network. We found that such treatments, despite only being administered to 1/3 of the bees, reduced trophallaxis among the entire group, indicating that changes in division of labor were associated with group-wide shifts in trophallaxis behavior. Despite these endocrine effects, there was no change in the predicted rate of information flow among members of the group, revealing that at least some aspects of the trophallaxis social network may be resilient to changes in division of labor. To our knowledge, these findings provide the first evidence that trophallaxis social dynamics are altered together with changes in division of labor in response to a socially relevant perturbation in honey bee colonies. They also suggest important new lines of study to explore the functional connections between trophallaxis social interactions and division of labor.

180

4.1 Introduction In both human and insect societies, division of labor is an emergent property (i.e. a result of the combined activities of components in a system) that arises in large part due to the social interactions between constituent members (Mersch 2016, Conradt and List 2009, Charbonneau, Blonder and Dornhaus 2013). Innate proclivities to perform specific behavioral tasks (based on heritable differences in response thresholds toward specific stimuli (Beshers, Robinson and Mittenthal 1999, Waibel et al. 2006, Page, Erber and Fondrk 1998, Page, Fondrk and Rueppell 2012)) are modified by the external and colony environments (Winston 1987, Robinson 1992, Jeanson et al. 2007, Robinson, Feinerman and Franks 2009) including social interactions (Beshers et al. 2001), resulting in task specialization by individuals within the society. Theoretically, this has the effect of enhancing the overall productivity and fitness of the social unit (Hölldobler and Wilson 1990, Mersch 2016, Brahma, Mandal and Gadagkar 2018).

In insect societies, the most fundamental form of division of labor occurs with respect to reproduction. Partially or completely sterile females (workers) contribute labor to colony maintenance and growth, thereby maximizing the reproductive potential of one or more reproductive females (queens). In many eusocial species, including honey bees, division of labor also occurs within the worker caste and it is this level of division of labor that seems to be largely responsible for increases in societal productivity related to colony maintenance and growth (Brahma et al. 2018). The temporal dynamics and flexibility of task specialization in workers can differ substantially depending on whether they arise from morphologically distinct castes of workers or between individuals of the same caste (see Chapter 1).

Adult worker honey bees exhibit an age-related polyethism during their 4-7 week lifespan. Worker behavior follows a stereotyped pattern of behavioral maturation, with bees performing different sets of tasks for days to weeks at a time, thus exhibiting long-term behavioral states and specializing in different tasks over the course of their lifespans (see Chapters 1-3). The fact that this pattern of maturation can be modulated or reversed by both external (social and abiotic) and internal (genetic and physiological) factors (Robinson 1992, Zayed and Robinson 2012, Page, Rueppell and Amdam 2012) allows the influence of these factors on division of labor to be dissected at the level of both the individual and the social group. Honey bees are thus an excellent model for determining how communication between colony members can contribute to the genesis and maintenance of division of labor within a society.

181

Honey bees possess multiple methods to communicate with one another. Broadly speaking, these modes of communication can be grouped into two types: indirect communication that broadcasts signals to the colony at large (such as volatile pheromones and nonvolatile pheromones that persist in an area for extended periods), and direct communication that occurs between spatiotemporally localized groups of bees (such as vibroacoustic or tactile stimuli and contact pheromones). The contributions of indirect communication to division of labor in numerous behavioral contexts, including the role of bee pheromones in regulating broodcare, foraging and aggressive behavior (Bartolotti & Costa 2014), are comparatively well understood.

Direct physical communication is known to be vital for task allocation (the direction and distribution of workloads between individuals with a specific behavioral specialization (Seeley et al. 1990). For example, contact pheromones have been implicated in regulating the age at which workers shift from working in the hive to foraging outside for nectar and pollen (Beshers et al. 2001, Leoncini et al. 2004, Muenz et al. 2012). However, the link between direct communication and division of labor is not as well characterized as for indirect communication. In particular, it is unclear how perturbations to the internal and external environments are transduced by direct communication to influence division of labor.

Trophallaxis is the directed exchange of food between two or more bees via their proboscides. Trophallaxis constitutes a sharing of information, at the bare minimum in the form of the food exchange itself (who has food and who needs food), and the number of trophallaxis interactions necessary to obtain nourishment is thought to directly influence division of labor (Seeley 1989). In addition, trophallactic events often occur at timescales that suggest that very little food is being exchanged and more often than if resource transfer was their sole purpose (Greenwald, Segre and Feinerman 2015), suggesting that trophallaxis has additional roles in communication. Indeed, proteomic analyses of the contents exchanged during trophallaxis have detected numerous signaling molecules (LeBoeuf et al. 2016), some of which are known to play a direct role in division of labor (Robinson, 1992).

Direct communication between individuals in a group can be represented as a social network whose topology can be probed for patterns in interactions at multiple levels (from individual social characteristics to cohorts to group-wide emergent properties). This approach has been successful in numerous animal systems (Croft et al. 2016) including social insects (Charbonneau et al. 2013),

182 and the use of social networks to explore division of labor in social insects is now a very active area of research (Mersch 2016).

Understanding how information is transmitted through trophallaxis networks in the context of division of labor may thus lead to fundamental insights into the way that workflows are structured in complex societies and how they adapt to changing internal and external states. Indeed, a previous study, conducted with a new automated detection system, found that honey bee trophallaxis networks are structured in ways that are similar to human social networks, but that unlike human-generated networks they are theoretically capable of facilitating the spread of information throughout the colony at short timescales (Gernat et al. 2018).

We therefore used this automated detection system (Gernat et al. 2018) to characterize patterns of trophallaxis within small (60 bee) groups and then study division of labor-related behavior. We generated both static and time-ordered (Holme and Saramaki 2012) networks of these trophallaxis interactions. Since previous studies have shown that laboratory assays can accurately identify behavioral states related to task performance (Li-Byarlay et al. 2014, Shpigler and Robinson 2015, Shpigler et al. 2017), we then combined these data with behavioral assays to link trophallaxis patterns with behavioral state. We hypothesized that differences in task-related behavior are connected to altered patterns of trophallaxis in the social network. To be as comprehensive as possible, we surveyed colony defense (conspecific aggression), broodcare, wax-building, fanning (used for thermoregulation and the spread of pheromones), and dorso- ventral abdominal vibrations (DVAV; a mode of modulatory communication thought to play a role in task allocation (Nieh 1998)). We also administered a hormone analog to shift the social demography of a subset of individuals within the group by accelerating behavioral maturation, hypothesizing that altering division of labor should perturb the rate of information flow and lead to the development of novel network states.

While division of labor has been linked to proximity-based social networks (Mersch, Crespi and Keller 2013), and trophallaxis social networks have been described for both bees (Gernat et al. 2018) and ants (Greenwald et al. 2015), to our knowledge no other study has attempted to alter both division of labor and trophallaxis social dynamics via a pharmacological manipulation.

4.2 Methods

183

4.2.1 Animals All experiments took place at the University of Illinois Bee Research Facility, Urbana, Illinois between the months of June and October in 2016. To minimize the effects of genetic variation on behavioral analyses, all experiments used adult worker bees from colonies headed by queens that had each been instrumentally inseminated with semen from a single drone. Due to haplodiploidy, this results in an average coefficient of relatedness of 0.75. Honeycomb frames containing pupae were removed from colonies one to two days prior to the beginning of each experiment and maintained in a dark incubator at 34◦ Celsius and 50% relative humidity. 0-24 h old adult workers were obtained from these frames, cold anesthetized, and tagged with “bcodes” (specially designed barcode-like identifiers) as described in (Gernat et al. 2018). The bees were then placed within a glass-walled observation hive containing a single one-sided honeycomb frame divided into nine sectors of equivalent size by wooden crossbars, with 60 bees placed in each sector. Instead of a queen, each sector had one queen equivalent of queen pheromone glued to the center (Intko Supply, Vancouver, British Columbia), and each was provisioned with 6mL of honey and pollen paste (45% honey, 45% pollen, 10% water). Bees were not able to move out of their own sector, physically communicate with bees of other sectors or leave the hive to forage outside, effectively creating nine micro-colonies within each observation hive. A total of 46 such sectors (contained within six observation hives) were used for this study.

4.2.2 Colony Observations Observations hives were housed in a dark room heated to 32◦C and maintained at 50% humidity. Images were captured at a rate of one frame per second over the course of the entire experiment, with infrared lighting illuminated the colony during periods of image capture from both the front and rear of the hive (via a blacklight). The glass window of the observation hive was changed periodically to prevent debris from obscuring the colony, but the hives were otherwise left undisturbed except during pharmacological treatments.

4.2.3 JHA Treatments After three days of observations, colonies were dismantled, and the bees from each sector were placed in plexiglass cages. The glass observation window was replaced with a transfer glass with removable portholes that could be centered over each sector, and the colony was placed in an area lit by a far-red LED light (Smart Vision Lights, Muskegon, Michigan USA). Since far red wavelengths are barely perceptible to bees, this allowed them to be transferred individually to the plexiglass cage via forceps without risk of escape. Sectors were then randomly assigned to one

184 of three categories: no treatment (NT), Juvenile Hormone analog (JHA) treatment, or acetone control (ACE) treatment (Figure 4.1a). Bees from NT sectors were cold-anesthetized in groups of three and randomly painted with two colors (Testor’s Corporation, Vernon Hill, Illinois USA). Bees from the JHA-treated sectors were also cold-anesthetized in groups of three; however, each bee in the group received a different treatment: one was treated topically with the JHA methoprene (200 ng/bee, a dose that is known to induce precocious foraging in bees (Robinson 1987)) dissolved in 1uL of acetone, one was treated with 1uL of acetone, and the third did not receive any treatment. This ensured that 1/3 of the sector was JHA treated, 1/3 was treated with acetone and 1/3 was untreated. ACE sectors were treated in the same manner as control sectors, but two of the three bees in each group received a topical treatment of 1uL acetone while the remaining bee was cold anesthetized but did not receive a treatment. This ensured that the same proportion of bees (2/3) were given a topical treatment in the JHA and ACE sectors. The bees in the JHA and ACE sectors were also painted with a color code designating their treatment and photographed, allowing their barcode identification to be linked to their treatment. After all the bees in a sector recovered from anesthetization, they were returned to their observation hive. Because JHA treatment is known to compress the broodcare phase of the worker division of labor schedule and cause precocious foraging (Robinson 1987; reviewed in Chapter 1), we hypothesized that this treatment would result in fewer bees capable of broodcare. This in turn was expected to alter division of labor, permitting us to determine whether patterns of trophallaxis interactivity are altered in the affected individuals and their nestmates.

4.2.4 Laboratory Division of Labor Assays On the eighth day of the experiment, colonies were dismantled as described previously and the bees were placed within plastic petri dishes with nine uniquely marked bees per dish, with an even distribution between treatment groups. The bottom of each petri dish was coated with wax foundation and was provisioned with 1mL of honey, 1mL of water and a 1g pollen ball. Two types of assays were performed; dishes were randomly assigned to aggression assays (simulating colony defense) or broodcare assays and allowed to acclimate to the testing environment for at least one hour. Broodcare assays were performed in a heated incubator (34◦C and 50% relative humidity) under light conditions, while aggression assays were performed in a temperature- controlled room (28◦C).

Assays were scored to develop an index of aggression and broodcare behavior as detailed in Chapter 3. In addition, fanning, wax-building and DVAV behaviors were catalogued during both

185 assays using scan sampling to give a more comprehensive picture of the bees’ behavioral state. One hour after the initial assay, the bees were tested a second time using the assay that they were not originally exposed to; each group was thus assayed for their responses to both broodcare- and aggression-inducing stimuli (Figure 4.1b). Bees that responded to either the broodcare or aggression stimulus but did not exhibit any of the behaviors described above (i.e. they did not nurse, exhibit aggression, fan, build wax, or perform DVAV). Those that did not respond to either stimulus in any way were labeled as ‘unresponsive’.

4.2.5 Image Processing and Trophallaxis Prediction Images were processed and analyzed as described in (Gernat et al. 2018). In brief, images were resized and sharpened, improving the rates of automated bcode and trophallaxis detection. The images were then converted into a binary format, and the location and orientation of each bee within the hive was inferred based on the identifiers present on each tag (using software developed by (Gernat et al. 2018)). After detection, bcodes were filtered to remove detections that could not be confidently read, that were duplicated or that corresponded with dead bees.

A machine vision algorithm then determined the shape and position of the tagged bee’s head in relation to the bcode. If an extended proboscis was detected between the heads of two tagged bees, it was considered a trophallaxis event. Consecutive images containing a trophallaxis event between the same bees were combined into a single interaction, and these raw interactions were filtered to remove spurious identifications (those shorter than 3 seconds or longer than three minutes). A previous study (Gernat et al. 2018) has found that even with 3 second interactions (the minimum amount of data) this method has excellent sensitivity (51%) and specificity (98%, with a 19% false positive rate, and 7% rate of false negatives). Since the vast majority of trophallaxis interactions are longer than three seconds, these metrics are nonetheless quite conservative for most interactions.

Information pertaining to these interactions (when and between whom they occurred, their duration, etc.) was stored in a series of undirected networks (i.e. without reference to which individual was the food donor and which the receiver), as detailed below.

4.2.6 Static Networks and Interaction Characteristics To assess the influence of JHA treatment on the interaction patterns of individual bees, we generated static networks for each day prior to and after the treatment. These networks allowed

186 us to determine the general interaction properties of each individual (including degree, number of interactions and the average interaction duration) in the sectors. We then used these properties to assess whether behavioral state was related to patterns of social interactivity in the trophallaxis network on the last two days of the experiment (See Section 4.2.8). We also compared changes in these properties between sectors, as well as between differentially treated individuals within the same sectors (i.e., JHA-treated vs NT controls).

4.2.7 Time-Ordered Networks and Information Flow To ascertain whether JHA treatments impacted the spread of information in the social groups, discrete time-ordered networks (Holme and Saramaki 2012) consisting of the two days pre- and post-treatment were generated for each group of bees. We then measured the rate of information flow within these networks using a susceptibility-infected model (Gernat et al. 2018). In brief, bees are selected at random as the starting points for an artificial signal. This signal is propagated from the initial bee to each bee it interacts with, infecting each in turn. These bees can then contribute to the signal’s spread, and so on. In principle this can be continued until the signal has been propagated to the entire social group, but the propagation asymptotes as it nears 100%. As a result, we used the time it requires to reach 20% of the target population as a cutoff point, as previous studies have done (Gernat et al. 2018, Karsai et al. 2011). This procedure is repeated using five hundred different trophallaxis interactions as the starting point, and the average of these simulations determines the speed of information transmission. This value was then normalized to a baseline established by repeating this procedure for 100 control networks (formed by randomly permuting the interaction times for each node (Gernat et al. 2018)). This normalized value represents the speedup (or slowdown) of information flow and is the metric we compared across networks.

4.2.8 Statistical Analyses Broodcare and aggression indexes were analyzed as detailed in Chapter 3 to determine how JHA affected division of labor within the groups, with one exception: because we were observing individual differences in trophallaxis interaction patterns we treated each individual bee in the group as an experimental unit rather than a pseudoreplicate. For wax-building, fanning and DVAV behaviors, the proportion of bees exhibiting each behavior was compared between the JHA- treated and NT bees using the Fisher Exact Test (with a Bonferroni correction). To simplify the analyses of individual node characteristics, the aggression and broodcare indices were also discretized into a binary format based on whether the bees performed a specific behavior,

187 allowing the values to be treated as categorical variables. In the instance of aggression, we classified the bees with an aggression index of 0.5 or higher (ca. the 90th percentile) as ‘aggressive’, as in Chapter 3.

Because node characteristics were right skewed, a Box-Cox transformation was applied (λ = 0.5). The data met assumptions of normality and homogeneity of variance post-transformation, allowing a factorial design to be used. Since bees could belong to more than one behavioral category, each behavior was treated as a separate main effect with two levels. No two-way or higher interaction effects were found to be significant between these variables, so only main effects were included in the final model and Type II Sums of Squares were used. Samples were additionally blocked by colony of origin and sector nested within date. A global Benjamini- Hochberg False Discovery Rate (FDR) correction was applied to all tests; FDR corrected p-values greater than 0.05 were not considered significant.

Information speedup was analyzed using non-parametric longitudinal models with the nparLD package in R (Noguchi et al. 2012). This approach allows for the factorial analysis of repeated measures data using a nonparametric rank-based statistical test. Importantly, unlike other nonparametric models for longitudinal two-way data (such as the Friedman test and extensions of the Kruskal-Wallis test), it also allows for an interaction term between time and the treatment variable.

4.3 Results 4.3.1 JHA Influences Behavioral State JHA is well known for its ability to induce precocious foraging in treated bees (Robinson 1987), and can reduce broodcare behavior under laboratory conditions (see Chapter 3). As expected, JHA treatment caused a significant reduction in both the broodcare index (as observed in Chapter 3; p < 0.01) and in the proportion of bees exhibiting broodcare behavior (p < 0.05, Figure 4.2). It also reduced the proportion of bees exhibiting wax-building behavior (p < 0.05, Figure 4.2). Although wax gland development has previously been shown to occur independent of Juvenile Hormone signaling (Muller and Hepburn 1994), in our case the behavior was performed exclusively in the broodcare context and a strong overlap between nurses and wax-builders was found (Fisher Exact Test p < 0.01), so this result is likely a side effect of the reduction in broodcare behavior by JHA administration. Unexpectedly, JHA also increased the incidence of vibration signaling (Figure 4.2), a previously unreported effect. However, it did not have an effect on the

188 aggression index (as in Chapter 3), or the proportion of bees exhibiting aggression, fanning, or social unresponsiveness (though in this case there was a trend, p < 0.1).

4.3.2 Interactivity in the Trophallaxis Social Network is Associated with Behavioral State We found that higher social interactivity in the trophallaxis network (the number of trophallaxis interactions, interaction partners (aka degree) and interaction duration) on the day prior to division of labor testing was associated with higher levels of behaviors that require interaction with nestmates. Specifically, the numbers of trophallaxis interactions and interaction partners were higher in individuals exhibiting broodcare, DVAV and aggression (Figure 4.3a). Increased trophallaxis interaction duration was also associated with increased broodcare and DVAV. These associations appear to be time dependent because some that were present one day before testing were not evident two days before testing (Figure 4.3a). Moreover, socially unresponsive bees had fewer trophallaxis interactions, interaction partners, and interacted for less time on average (Figure 4.3a). Although a previous study assaying social sensitivity used two repetitions of the broodcare and aggression assays to determine whether bees were unresponsive (Shpigler et al. 2017), the total proportion of bees identified as such in both studies is remarkably similar (ca. 14% in the previous study and ca. 16% in the current study).

Investigating trophallaxis social interactivity during the nighttime (Figure 4.3b) revealed similar patterns for bees that exhibited broodcare (which is known to occur around the clock (Winston 1987), DVAV (which is known to occur in the hive during resting hours (Nieh 1998)) and social unresponsiveness (which implies that this may be a general state of social inactivity). By contrast, aggressive behavior was associated with baseline levels of trophallaxis social interactivity on the final night of the assay (Figure 4.3b). Since it is thought that aggressive bees (such as guards and soldiers) are more likely to be quiescent at night, this may be an indication of the onset of circadian rhythmicity in these groups.

4.3.3 JHA Perturbation Reduces Interactivity in the Trophallaxis Social Network JHA treatment significantly reduced the number of interactions and interaction partners for trophallaxis, and the duration of interactions in the JHA-treated sectors (Figure 4.4a). The duration of interactions was also reduced, but the presence of differences between the NT and JHA sectors prior to the treatment makes this finding suspect (Figure 4.4a). No effect of handling was observed in the NT treated sectors. However, when comparing the effect of JHA, NT and ACE treatment on individuals within the treated sectors, no significant differences between the treatments were

189 found for any of the tested characteristics (Figure 4.4b). Therefore, to rule out the possibility that the observed group-level differences were due to ACE rather than JHA, we included additional sectors where 1/3 of the bees belonged to the NT group and 2/3 to the ACE treated groups. Doing so revealed no group-wise differences between the NT- and ACE-treated sectors, indicating that the JHA treatment was the critical factor in altering the number of interactions and interaction partners.

4.3.4 JHA Perturbation Does Not Alter Information Flow Although the datasets we used to generate our networks were substantially smaller than those previously used to study information flow (Gernat et al. 2018), we observed a similar speedup in the theoretical transmission of information relative to the randomized control networks in every network (Figure 4.5a), suggesting that this property may be size invariant. Moreover, JHA administration had no effect on the degree of network speedup observed (Repeated Measures ANOVA, Figure 5a), indicating that the speedup is resilient to endocrine mediated alterations of division of labor.

As honey bees age, their activities begin to exhibit a circadian rhythm even in the absence of environmental cues. To determine whether these rhythms influenced information flow, we compared rates of information flow during the day (sunrise to sunset) and night (sunset to sunrise). However, we found no differences in information flow as a function of time of day. Similarly, there was neither a significant main effect of JHA treatment on information flow, nor a significant interaction between treatment and the time factor (Figure 4.5b).

4.4 Discussion By finding that bees with distinct behavioral states exhibit differences in trophallaxis interaction patterns, we show for the first time that division of labor is associated with the properties of a trophallaxis social network, and that the two are quite probably functionally linked. It is reasonable that tasks that necessitate interactions with nestmates (such as colony defense, broodcare and vibration signaling) are associated with increases in social interactivity in the trophallaxis social network, whereas social unresponsiveness is linked to a reduction in the number of trophallaxis interactions and interaction partners.

Perturbing social groups using a hormone analog altered the trophallaxis interaction patterns of individuals within the groups, reducing the number of interactions and interaction partners of bees

190 within the treated sectors. This indicates that modulating the behavioral states of individual bees can induce group-wide changes in social interactivity in the trophallaxis network. However, the fact that this perturbation was transient in nature (lasting only a single day) is evidence for resiliency in the trophallaxis network, a characteristic that has been reported in insect social networks in multiple contexts (Gernat et al. 2018, Middleton and Latty 2016, Naug 2009).

It is also intriguing that there were no differences in trophallaxis interaction properties observed between differentially treated bees within the same sector. That is, treating a subset of the bees in a group with JHA altered the social interactivity of every bee (even those that were not treated with JHA) in the same manner. The fact that untreated individuals within the same group do not compensate for the decrease in trophallaxis by treated individuals may have dramatic ramifications for the function of the trophallaxis network, as it indicates that there may not be a setpoint or equilibrium state for overall social activity at the group level. It will be interesting to determine whether this rule holds for different kinds of perturbations, or for perturbations of division of labor in different social networks (including those of different species).

Although we found that division of labor is associated with the trophallaxis interaction properties of individual bees, and that modulating behavioral state with an endocrine analog can alter these same properties at the group level, we did not find an effect of JHA on information spreading. The fact that information spreading in small groups of bees exhibited the same information speedup (relative to randomized control networks) as in much larger trophallaxis networks monitored for longer timescales (Gernat et al. 2018) suggests that this property is largely scale invariant. Moreover, the lack of any apparent differences in information speedup across days, based on circadian rhythmicity or in response to the removal of foragers (Gernat et al. 2018) or hormone manipulations that otherwise influenced the social properties of individual nodes within the network implies that this speedup may be a fundamental property of trophallaxis interactions. From an ethological perspective, this is very compelling, as it would suggest that trophallaxis networks may be structured to increase the rate of information flow within the colony at a very basic level, possibly increasing the colony’s adaptability to changes in external and internal state.

Because differences in task-related behavior are associated with differences in interactivity within the trophallaxis social network, we posit that there is a connection between division of labor and patterns of trophallaxis behavior. Moreover, we show for the first time that altering division of labor using JHA results in concomitant changes in trophallaxis at the level of the entire group, indicating

191 that shifts in task specialization can modulate group level social dynamics. However, alterations in division of labor do not appear to affect the (potentially fundamental) speedup of information flow, suggesting that only specific levels of network properties may be related to task specialization.

4.5 Future Directions Using small group sizes facilitated the observation of a large number of groups under laboratory conditions. This provided enough statistical power to allow for the analysis of group-wide information flow across multiple treatments and providing more control over experimental conditions than is achievable in full colonies with access to the external environment. However, this experimental paradigm suffers from two caveats. First, we were not able to test for foraging behavior; it is therefore possible that the socially unresponsive bees have been conflated with foragers. In addition, the fact that some differences in interaction properties are not present prior to the day before behavioral testing suggests that these differences may not be as stable as one would expect, given that behavioral states can be stable for days or even weeks in worker honey bees. However, the fact that these are relatively young social groups could indicate that their division of labor is not stable yet and may be in a greater state of flux than in actual colonies.

Additional research should therefore focus on extended observation of colonies with access to the external environment. This would allow us to classify how foraging behavior contributes to the social network, as well as to properly determine whether individual differences in trophallaxis social interactivity are persistent and whether they can be predictive of behavioral states. In particular, it may be fruitful to probe the long-term behavior of unresponsive workers in the trophallaxis social network. Although the purpose of unresponsive workers (alternately labeled “inactive” or “lazy”) in the colony is unknown, they occur in many social insect species, and it has been suggested that they are a reserve force of workers (Charbonneau, Sasaki and Dornhaus 2017). There is also evidence that a continuum of response thresholds are important for the long- term survival of the colony (Hasegawa et al. 2016). Understanding whether and how they contribute to colony adaptability through shifts in division of labor and social networks may thus provide insight into common mechanisms for generating robustness and resilience in societies.

It also remains to be determined whether patterns of trophallaxis interaction help give rise to division of labor or merely result from differences in the behavioral repertoires of individual bees. To ascertain whether a causal relationship between trophallaxis and division of labor exists, future

192 experiments will need to examine interaction patterns in individuals on the cusp of transitioning to a new behavioral state. Additional analytical techniques (such as algorithms designed to find temporal motifs and interaction patterns between bees with different behavioral states) will need to be brought to bear to properly define the relationship between changes in division of labor and information flow within the colony. Indeed, since previous studies indicate that broodcare (Whitfield, Cziko and Robinson 2003, Khamis et al. 2015), DVAV (Chandrasekaran et al. 2011) and social sensitivity (Shpigler et al. 2017) are associated with distinct brain transcriptomic signatures, it may be possible for future studies to dissect the link between behavioral state, transcriptomic state and patterns of trophallaxis.

This chapter therefore represents a promising start toward characterizing the relationship between division of labor and social interactions in honey bee trophallaxis networks. However, further work is needed to properly assess the extent of this relationship and its importance for colony productivity and adaptability in response to changing internal and external states.

Acknowledgements I would like to acknowledge and thank Tim Gernat, Vikyath D. Rao and Nigel Goldenfeld for their extensive contributions to the analytical techniques and computer programs used to generate the trophallaxis social networks. I would also like to thank Allyson M. Ray and Sarai H. Stuart for their assistance with field work and behavioral assays, as well as Terry Harrison and Alison Sankey for managing the honey bee colonies. This research was made possible by grant # 5R01GM117467-02 from the National Institutes of Health.

193

4.6 Figures

Figure 4.1. Experimental Design. A) Bees were tagged with unique bcodes and placed within a glass-walled observation hive divided into nine discrete sectors (1). They were monitored for three days, and then sectors were randomly assigned to receive NT (all bees handled but untreated), ACE (two of every three bees treated with acetone and the third untreated) or JHA (one bee treated with JHA and one of each control) treatments (2) and returned to the colony for two additional days of observation (3). B) After observations were complete, bees from each sector were divided into groups of nine, with proportional representation of each treatment in the sector and placed in petri dishes. The dishes were then exposed to either the broodcare or aggression assay (selected at random), and one hour later were tested using the other assay.

194

Figure 4.2: JHA Alters Division of Labor. Treating with JHA reduced the proportion of bees exhibiting broodcare and wax-building behavior and increased the proportion exhibiting DVAV (a hitherto unreported effect) relative to nestmates in the NT control group (Fisher Exact Test, p<0.05, Bonferroni corrected). However, it did not have a significant influence on the proportion of aggressive, fanning or socially unresponsive (those that did not respond to either the broodcare or aggression stimulus) bees.

195

* FDR < 0.05

Figure 4.3: Behavioral State is Associated with Interactivity in the Trophallaxis Social Network. A) Interactivity in the trophallaxis network for the two days prior to behavioral testing for bees that exhibited fanning (n=150), wax-building (n=53), aggression (n=88), broodcare (n=45), or DVAV (n=27) behaviors, or that were socially unresponsive (n=107). Bees that did not belong to any of these categories are displayed for references as the “Background” group (n=317), and all values (y-axis) are normalized to this group’s mean. Asterisks designate that a significant difference was detected for the main effect associated with the given behavior (FDR < 0.05). B) Interactivity in the trophallaxis network for the two nights prior to behavioral testing for the same groups as in A. As above, all values are normalized to the “Background” group, and asterisks

196 designate that a significant difference was detected for the main effect associated with the given behavior (FDR < 0.05).

197

Figure 4.4: JHA Influences Intergroup but not Intragroup Interactivity in the Trophallaxis Social Network. A) The number of trophallaxis interactions, interaction partners and interaction

198 duration of differentially treated sectors for the two days prior to (-) and after (+) treatment (denoted by the vertical bar). A significant interaction effect between treatment and time was found in all three instances (Repeated Measures ANOVA, p < 0.001), with post-hoc analysis revealing significant differences between the day prior to and after treatment for the JHA treatment relative to the NT control (Tukey’s HSD, p < 0.001). Note that a significant difference exists for interaction duration prior to the administration of the JHA treatment (Tukey’s HSD, p < 0.001). B) However, there were no discernable differences in the way that NT-, ACE- and JHA-treated individuals in the JHA-treated sectors responded to the treatment (a significant reduction in all three measures on the first day after the treatment, followed by a rebound on the second day, both significant (Tukey’s HSD, p <0.01)).

199

Figure 4.5: Information Flow in Sector Trophallaxis Networks is Faster than Random, but Unaffected by JHA Perturbation. A) Representative depiction of information flow in a temporal network containing 60 bees. The black line represents the proportion of bees reached (y-axis) over time (x-axis) in network describing a sector’s trophallaxis behavior. The pink lines represent the 100 randomized control networks. The network speedup is determined by the amount of time it takes to for the signal to reach 20% of the bees in the experimental network normalized to the mean time it takes for the randomized control networks to do the same. B) Neither the JHA

200 treatment nor either control condition influenced network speedup in a reliable manner. Lines represent the change in network speedup before and after the perturbation for each sector in the NT Control (n = 17), ACE control (n = 8) and JHA treatment (n = 18) conditions. Note, however that network speedup was positive in all instances, indicating that the actual trophallaxis network transmitted the signal faster than the randomized control, as previously reported (Gernat et al. 2018).

201

References Beshers, S. N., Z. Y. Huang, Y. Oono & G. E. Robinson (2001) Social inhibition and the regulation of temporal polyethism in honey bees. Journal of Theoretical Biology, 213, 461-479. Beshers, S. N., G. E. Robinson & J. E. Mittenthal (1999) Response thresholds and division of labor in insect colonies. Information Processing in Social Insects, 115-139. Brahma, A., S. Mandal & R. Gadagkar (2018) Emergence of cooperation and division of labor in the primitively eusocial wasp Ropalidia marginata. Proceedings of the National Academy of Sciences of the United States of America, 115, 756-761. Chandrasekaran, S., S. A. Ament, J. A. Eddy, S. L. Rodriguez-Zas, B. R. Schatz, N. D. Price & G. E. Robinson (2011) Behavior-specific changes in transcriptional modules lead to distinct and predictable neurogenomic states. Proc Natl Acad Sci U S A, 108, 18020-5. Charbonneau, D., B. Blonder & A. Dornhaus. 2013. Social Insects: A Model System for Network Dynamics. In Temporal Networks, eds. P. Holme & J. Saramäki, 217-244. Berlin, Heidelberg: Springer Berlin Heidelberg. Charbonneau, D., T. Sasaki & A. Dornhaus (2017) Who needs 'lazy' workers? Inactive workers act as a 'reserve' labor force replacing active workers, but inactive workers are not replaced when they are removed. Plos One, 12. Conradt, L. & C. List (2009) Group decisions in humans and animals: a survey. Philosophical Transactions of the Royal Society B-Biological Sciences, 364, 719-742. Gernat, T., V. D. Rao, M. Middendorf, H. Dankowicz, N. Goldenfeld & G. E. Robinson (2018) Automated monitoring of behavior reveals bursty interaction patterns and rapid spreading dynamics in honeybee social networks. Proceedings of the National Academy of Sciences of the United States of America, 115, 1433-1438. Greenwald, E., E. Segre & O. Feinerman (2015) Ant trophallactic networks: simultaneous measurement of interaction patterns and food dissemination. Scientific Reports, 5. Hasegawa, E., Y. Ishii, K. Tada, K. Kobayashi & J. Yoshimura (2016) Lazy workers are necessary for long-term sustainability in insect societies. Scientific Reports, 6. Holme, P. & J. Saramaki (2012) Temporal networks. Physics Reports-Review Section of Physics Letters, 519, 97-125. Hölldobler, B. & E. O. Wilson. 1990. The ants. Cambridge, Mass.: Belknap Press of Harvard University Press. Jeanson, R., J. H. Fewell, R. Gorelick & S. M. Bertram (2007) Emergence of increased division of labor as a function of group size. Behavioral Ecology and Sociobiology, 62, 289-298. Karsai, M., M. Kivela, R. K. Pan, K. Kaski, J. Kertesz, A. L. Barabasi & J. Saramaki (2011) Small

202

but slow world: How network topology and burstiness slow down spreading. Physical Review E, 83. Khamis, A. M., A. R. Hamilton, Y. A. Medvedeva, T. Alam, I. Alam, M. Essack, B. Umylny, B. R. Jankovic, N. L. Naeger, M. Suzuki, M. Harbers, G. E. Robinson & V. B. Bajic (2015) Insights into the Transcriptional Architecture of Behavioral Plasticity in the Honey Bee Apis mellifera. Scientific Reports, 5. LeBoeuf, A. C., P. Waridel, C. S. Brent, A. N. Goncalves, L. Menin, D. Ortiz, O. Riba-Grognuz, A. Koto, Z. G. Soares, E. Privman, E. A. Miska, R. Benton & L. Keller (2016) Oral transfer of chemical cues, growth proteins and hormones in social insects. Elife, 5. Leoncini, I., Y. Le Conte, G. Costagliola, E. Plettner, A. L. Toth, M. W. Wang, Z. Huang, J. M. Becard, D. Crauser, K. N. Slessor & G. E. Robinson (2004) Regulation of behavioral maturation by a primer pheromone produced by adult worker honey bees. Proceedings of the National Academy of Sciences of the United States of America, 101, 17559-17564. Li-Byarlay, H., C. C. Rittschof, J. H. Massey, B. R. Pittendrigh & G. E. Robinson (2014) Socially responsive effects of brain oxidative metabolism on aggression. Proceedings of the National Academy of Sciences of the United States of America, 111, 12533-12537. Mersch, D. P. (2016) The social mirror for division of labor: what network topology and dynamics can teach us about organization of work in insect societies. Behavioral Ecology and Sociobiology, 70, 1087-1099. Mersch, D. P., A. Crespi & L. Keller (2013) Tracking Individuals Shows Spatial Fidelity Is a Key Regulator of Ant Social Organization. Science, 340, 1090-1093. Middleton, E. J. T. & T. Latty (2016) Resilience in social insect infrastructure systems. Journal of the Royal Society Interface, 13. Muenz, T. S., A. Maisonnasse, E. Plettner, Y. Le Conte & W. Rossler (2012) Sensory reception of the primer pheromone ethyl oleate. Naturwissenschaften, 99, 421-425. Muller, W. J. & H. R. Hepburn (1994) Juvenile-hormone-III and wax secretion in honeybees (Apis- mellifera-capensis). Journal of Insect Physiology, 40, 873-881. Naug, D. (2009) Structure and resilience of the social network in an insect colony as a function of colony size. Behavioral Ecology and Sociobiology, 63, 1023-1028. Nieh, J. C. (1998) The honey bee shaking signal: function and design of a modulatory communication signal. Behavioral Ecology and Sociobiology, 42, 23-36. Noguchi, K., Y. R. Gel, E. Brunner & F. Konietschke (2012) nparLD: An R Software Package for the Nonparametric Analysis of Longitudinal Data in Factorial Experiments. Journal of Statistical Software, 50, 1-23.

203

Page, R. E., J. Erber & M. K. Fondrk (1998) The effect of genotype on response thresholds to sucrose and foraging behavior of honey bees (Apis mellifera L.). Journal of Comparative Physiology a-Neuroethology Sensory Neural and Behavioral Physiology, 182, 489-500. Page, R. E., Jr., M. K. Fondrk & O. Rueppell (2012) Complex pleiotropy characterizes the pollen hoarding syndrome in honey bees (Apis mellifera L.). Behavioral Ecology and Sociobiology, 66, 1459-1466. Page, R. E., Jr., O. Rueppell & G. V. Amdam (2012) Genetics of reproduction and regulation of honeybee (Apis mellifera L.) social behavior. Annu Rev Genet, 46, 97-119. Robinson, E. J. H., O. Feinerman & N. R. Franks (2009) Flexible task allocation and the organization of work in ants. Proceedings of the Royal Society B-Biological Sciences, 276, 4373-4380. Robinson, G. E. (1987) Regulation of honey-bee age polyethism by juvenile-hormone. Behavioral Ecology and Sociobiology, 20, 329-338. --- (1992) Regulation of division of labor in insect societies. Annual Review of Entomology, 37, 637-665. Seeley, T. D. (1989) SOCIAL FORAGING IN HONEY BEES - HOW NECTAR FORAGERS ASSESS THEIR COLONY NUTRITIONAL-STATUS. Behavioral Ecology and Sociobiology, 24, 181-199. Seeley, T. D., S. Camazine, J. Sneyd, G. K. Veeresh, B. Mallik & C. A. Viraktamath. 1990. Collective decision-making in honey bees: how colonies choose among nectar sources. Shpigler, H. Y. & G. E. Robinson (2015) Laboratory Assay of Brood Care for Quantitative Analyses of Individual Differences in Honey Bee (Apis mellifera) Affiliative Behavior. PloS one, 10, e0143183-e0143183. Shpigler, H. Y., M. C. Saul, F. Corona, L. Block, A. C. Ahmed, S. D. Zhao & G. E. Robinson (2017) Deep evolutionary conservation of autism-related genes. Proceedings of the National Academy of Sciences of the United States of America, 114, 9653-9658. Waibel, M., D. Floreano, S. Magnenat & L. Keller (2006) Division of labour and colony efficiency in social insects: effects of interactions between genetic architecture, colony kin structure and rate of perturbations. Proceedings of the Royal Society B-Biological Sciences, 273, 1815-1823. Whitfield, C. W., A. M. Cziko & G. E. Robinson (2003) Gene expression profiles in the brain predict behavior in individual honey bees. Science, 302, 296-299. Winston, M. L. 1987. The Biology of the Honey Bee. USA: Harvard University Press. Zayed, A. & G. E. Robinson (2012) Understanding the Relationship Between Brain Gene

204

Expression and Social Behavior: Lessons from the Honey Bee. Annual Review of Genetics, Vol 46, 46, 591-615.

205

Chapter 5: Afterword Division of labor is crucial to the function of complex societies, enhancing group adaptability and productivity in the face of changing internal and external conditions. The work presented in this dissertation reviews and extends the theoretical (Chapter 1), transcriptomic (Chapters 2 and 3), and social (Chapter 4) underpinnings of division of labor.

The existence of commonalities in the way that division of labor arises and is structured in complex societies across disparate taxa from insects to primates strongly suggests that these properties are highly adaptive and a remarkable example of convergent evolution. By extension, a comparative approach can likely help to elucidate general heuristics governing division of labor. For instance, as noted in Chapter 1 similar endocrine systems have been coopted to regulate division of labor across distinct origins not just within the Hymenoptera but the Isoptera as well. Therefore, while honey bees were used as the principal animal model in the studies presented here, the results may be applicable to a broad range of social species.

Although there is a strong association between brain transcriptomic state and long-term behavioral states such as task specialization, the regulatory rules governing this relationship are not well defined. Chapter 2 probed the transcriptional regulatory architecture underlying division of labor and linked canonical regulators of neural activity to long-term behavioral states associated with honey bee division of labor. The findings presented indicated that five of these TFs (NF-κB, egr, pax6, hairy, and clockwork orange) form a co-regulatory module that acts upon more than half of the genes upregulated in foragers. While the specific regulators involved in division of labor may be species-specific, properties of key TFs and how they interact to govern downstream transcription may be generalizable to other systems. Future studies should therefore search for similar modules of co-regulators governing division of labor. They should also attempt to ascertain how the components of such modules interact (i.e., how many regulators are concurrently bound to a promoter in different conditions, and whether there are tissue specific differences in binding) to give rise to behaviorally related gene expression.

Since prior work has shown that individual honey bees that alter task specialization in response to the needs of a society undergo dramatic changes in brain transcriptomic state, it’s reasonable to question whether the transcriptional regulatory architecture can itself exhibit plasticity in relation to division of labor. Chapter 3 addressed this question using a combination of bioninformatic analyses and empirical experiments across several socially relevant contexts. It identified a small

206 subset of TFs that exhibited signatures of brain regulatory network plasticity as a function of behavioral state. Intriguingly, the TFs so identified were either pleiotropic signal integrators (in the case of Broad and NF-κB) or epigenetic regulators (in the case of TRL and Yl-1), implying that the general function of a TF may be related to the flexibility of its TF-target relationships. Additionally, the fact that Broad appears to influence at least some behavioral states independently of the downstream TF Ftz-F1 suggests that one path for a pleiotropic regulator to activate context-dependent transcriptional cascades.

Division of labor is a key social trait of honey bee colonies, and so is trophallaxis, the exchange of food and oral secretions. The findings presented in Chapter 4 indicate that division of labor is associated with specific patterns of trophallaxis, and that disruptions to division of labor can influence trophallaxis networks, even in individuals that were not directly affected by the disruption. This suggests that there may be causal relationship between group-wide social interactions and division of labor. Moreover, specific interaction patterns may be predictive of behavioral state; if so, it may be possible to infer shifts in division of labor simply by monitoring changing patterns of trophallactic interaction between individuals in the society. Discovering and modeling these dynamics in honey bees and other species would provide essential information about social mechanisms associated with division of labor, perhaps some causal.

These chapters thus contribute to understanding division of labor on multiple discrete levels, from the transcriptomes of individual bees to group social dynamics. A comprehensive understanding of division of labor will require integrative experiments that bridge the gap between these different experimental scales. For instance, understanding whether and how transcriptional networks and social networks operate by similar rules and feed into one another could provide much more holistic and accurate models of how division of labor functions. Only when the transcriptional, physiological and behavioral state of each individual in a society is viewed collectively and placed in a social and evolutionary context can a truly comprehensive framework of division of labor be formed.

207

Appendix A: Supplementary Figures

Figure A.1. Coverage depth saturation curve for all genes with detectable levels of expression in nurses and foragers. RSeQC calculates the relative error for each gene using specified proportions (subsets) of the reads assigned to that gene. These subsets are represented in the x- axis, while the y-axis shows how the estimated distribution of the resampled data deviates from real expression values. The subplots are divided based on the level of expression of their constituent genes (Q1 – lowest 25%, Q4 – highest 25%). Regardless of the level of expression, the percent relative error converges on a stable and accurate representation of the true data, indicating that the read coverage was adequate for differential expression analyses.

208

Figure A.2. Multidimensional Scaling (MDS) plot of gene expression profiles for all samples shows a higher degree of variation within forager samples than within nurse samples. The distance on the plot between each pair of samples represents the biological coefficient of variation BCV (the square root of the dispersion parameter under the negative binomial distribution for the 500 genes with the largest absolute log-fold-changes of expression between samples).

209

Figure A.3. Heatmap for the hierarchical clustering of the gene expression profiles of 16 honey bees. Rows correspond to the 12,453 genes with detectable levels of expression. Columns represent samples. The clustering identified two separate groups of transcription profiles, supporting the clear distinction between the two age-related behavioral groups, nurses and foragers.

210

Figure A.4. Heatmap for the hierarchical clustering of the gene expression profiles of 16 honey bees. Rows correspond to the 1,058 DEGs and columns represent samples. The clustering shows a clear separation in gene expression pattern between nurses and foragers.

211

Figure A.5. Plot showing the relationship between differential gene expression and read coverage. Differentially expressed genes (FDR<0.05) are in red and non-differentially expressed genes are in black. The two horizontal blue lines show 4-fold changes (log-FC of 2).

212

Figure A.6. Heatmap for the hierarchical clustering of the expression data of the 239 honey bee TFs with detectable levels of brain expression. Rows represent the 239 TF genes and columns represent nurse (labeled ‘N’) and forager (labeled ‘F’) samples.

213

Figure A.7. Heatmap for the hierarchical clustering of the expression data of 26 differentially expressed honey bee Apis mellifera TF genes. 22 TF genes are upregulated in foragers, and 4 TF genes are upregulated in nurses. Rows represent the 26 TF genes. Columns represent nurse (‘N’) and forager (‘F’) samples.

214

Figure A.8. Sequence logos for the position weight matrix (PWM) of the five TFs (hes1, pax6, nf- kb, egr and clockwork orange) predicted to co-regulate nearly half of all forager upregulated genes. The TFs egr and clockwork orange have three and two PWMs, respectively. As shown the logos, the motifs of many of these TFs are G/C rich.

215

Figure A.9. Boxplots for the expression levels of the five TFs (hes1, pax6, nf-kb, egr and clockwork orange) predicted to co-regulate nearly half of the forager upregulated genes.

216

Figure A.10. Identification of gene TSSs and related promoters. CAGE paired-end reads were mapped to the reference genome and viewed in reference to the newest honey bee gene annotation (OGS v3.2) to provide accurate identification of gene TSSs and related promoters. We identified different types of transcription start sites (TSS) based on the number of TSSs. Some genes have multiple TSSs (A) (gene ID: GB55365) while other genes use single TSSs (B) (gene IDs: GB40769 and GB40770).

217

Figure A.11. Effect of analysis technique on alternative TSS identification. Displays the influence of analysis technique on the identification of genes with alternative TSSs, using CAGE clusters vs. individual tags and with or without requiring a mutual distance (M.D.) of 100 bp.

218

Figure A.12. RNAi efficacy in bees from colonies used for aggression assays. broad and ftz-f1 RNAi treatments were effective in reducing Broad and Ftz-F1 expression, respectively, in all but one colony (2013-3) during aggression testing in 2013 (A and B) and 2014 (C; ANOVA, p < 0.05). The y-axis represents gene expression of broad RNAi-treated samples relative to the average expression of gfp RNAi-treated control samples (n ~ 50 per colony). In all box plots, center lines show the group median and x’s indicate the group means; the limits represent the 25th and 75th percentiles, and whiskers extend up to 1.5 times the interquartile range from these percentiles.

219

Figure A.13. RNAi efficacy in bees from two different colonies used for brood care assays. broad (A) and ftz-f1 (B) RNAi were effective in reducing Broad and Ftz-F1 expression, respectively, in all but one brood care trial, which showed a marginal, non-significant trend (p < 0.1). The effects of RNAi were also at least partially rescued by JHA treatments in all but that one experiment. Letters denote significantly different groups (ANOVA, n ~ 16 per group, p < 0.05). In all box plots, center lines show the group median and x’s indicate the group means; the limits represent the 25th and 75th percentiles, and whiskers extend up to 1.5 times the interquartile range from these percentiles.

220

Figure A.14. Diminished efficacy of RNAi treatment in trials conducted in 2015 relative to 2013 and 2014. (A) RNAi was effective in the vast majority of trials performed in 2013 (11 of 14) and 2014 (5 of 6). However, in RNAi trials performed in 2015 (which were performed as part of a different study not reported in this manuscript), a successful knockdown was obtained in only one of three trials. (B) Together with a marked decline in the average knockdown percentage of successful knockdowns, this points to a sudden reduction in overall RNAi efficacy over a three year period.

We speculate that the reason for this decline might be related to world-wide declines in bee health, but the precise mechanism is unknown. Although we have yet to discover the cause underlying the decrease in RNAi efficiency that we observed in 2015, the most promising explanation relates to viral infection. Honey bees are susceptible to a number of viral pathogens, some of which have functional or putative viral suppressors of RNAi systems (Brutscher, McMenamin, & Flenniken, 2016; Ryabov et al., 2014). The decrease in efficacy is also concurrent with global increases in honey bee virus pathogenicity (Wilfert et al.). The dynamics of viral load and pathogenicity in bees are complicated and poorly understood. Susceptibility appears to be a confluence of factors, and some bees can tolerate high viral titers without exhibiting symptoms (de Miranda & Genersch). Thus, although RNAi in honey bees has been effectively administered to adult bees by multiple laboratories including our own using abdominal (Ament et al., 2012; Ament et al., 2011; Guidugli et al., 2005; Wang, Brent, Fennern, & Amdam, 2012) and brain (Farooqui, Robinson, Vaessin, & Smith, 2003; Müßig et al., 2010; Rein, Mustard, Strauch, Smith, & Galizia, 2013) injections, decreased efficacy of RNAi has emerged as a challenge to be addressed in future research.

221

Figure A.15. broad RNAi influenced the expression of Broad and Ftz-F1 target genes. broad RNAi reduced the expression of both Broad and its targets (A) and Ftz-F1 and its targets (B), suggesting that Ftz-F1 is downstream of Broad-initiated transcriptional cascades. The y-axis represents gene expression of the broad RNAi treated samples (n = 51) relative to the average expression of the gfp treated control samples (n = 52). In all box plots, center lines show the group median and x’s indicate the group means; the limits represent the 25th and 75th percentiles, and whiskers extend up to 1.5 times the interquartile range from these percentiles. Data depicted are from 2013.

222

Figure A.16. ftz-f1 RNAi altered the expression of Ftz-F1 target genes, but not Broad target genes. ftz-f1 RNAi reduced the expression of Ftz-F1 and its target genes (A), but not Broad or its targets (B), suggesting that Broad and Ftz-F1 do not have reciprocally regulate one another. The y-axis represents gene expression of the broad RNAi treated samples (n = 75) relative to the average expression of the gfp treated control samples (n = 78). In all box plots, center lines show the group median and x’s indicate the group means; the limits represent the 25th and 75th percentiles, and whiskers extend up to 1.5 times the interquartile range from these percentiles. Data depicted are from 2013.

223

Figure A.17. Influence of RNAi and JHA on aggression across background colonies. (A) broad RNAi significantly reduced aggression in 2 out of the 3 colonies tested in 2013. The one colony that did not show this effect (2013-3) was the same colony that did not exhibit a significant knockdown of Broad expression in response to broad RNAi. (B) As predicted, ftz-f1 RNAi did not significantly alter aggressive behavior in any of the colonies. (C) The effect of broad RNAi was replicated in 2014 across all three colonies tested, and this effect was partially rescued by the JHA in 2 out of 3 colonies. Letters denote significantly different groups (ANOVA, p < 0.05). In all box plots, center lines show the group median and x’s indicate the group means; the limits represent the 25th and 75th percentiles, and whiskers extend up to 1.5 times the interquartile range from these percentiles.

224

Figure A.18. Influence of RNAi and JHA on brood care behavior across background colonies. (A) broad RNAi increased nursing behavior in 3 out of 3 colonies. (B) ftz-f1 RNAi increased nursing behavior in 2 out of 3 colonies. JHA at least partially rescued this effect in all of the colonies that also had a significant RNAi treatment effect. Letters denote significantly different groups (ANOVA, p < 0.05). In all box plots, center lines show the group median and x’s indicate the group means; the limits represent the 25th and 75th percentiles, and whiskers extend up to 1.5 times the interquartile range from these percentiles.

225

Figure A.19. broad RNAi revealed time-dependent effects of Broad on target genes. Analyzing the transcriptomic response of bees exposed to an intruder as a time series (0, 60, 120 minutes post-assay) revealed that Broad’s relationship with its target genes changed rapidly as a function of a social stimulus (ANCOVA, p < 0.05). This is further evidence for TRN rewiring in response to social stimuli.

226

Figure A.20 RUVcorr corrects for batch-wise variation prior to ANCOVA. (A) Principal Component Analysis revealed substantial batch effects between microarray datasets, which were conflated with the behavioral contexts assayed. (B) To prevent this from affecting the ANCOVAs, we applied the RUVcorr package, which largely eliminated the batch effects along the first two principal components.

227

Figure A.21. Effect of RNAi injection on mortality. Low mortality was observed on the day of behavioral testing (48 hours after RNAi injection) for experiments contrasting br and gfp RNAi (A), and ftz-f1 and gfp RNAi (B). No significant differences in mortality were found between RNAi or JHA treatment groups (Kruskal-Wallis test).

228

Figure A.22. Broad RNAi has reduced influence on the expression of CG17912 target genes. To explore whether broad RNAi had nonspecific effects on gene expression, we tested whether five putative target genes of the TF CG17912 responded to broad RNAi treatments. The y-axis represents gene expression of the broad RNAi-treated samples (n = 51) relative to the average expression of the gfp treated control samples (n = 52). Only 1 out of 5 genes was significantly affected by broad RNAi (p < 0.05), providing some indication of specificity. In all box plots, center lines show the group median and x’s indicate the group means; the limits represent the 25th and 75th percentiles, and whiskers extend up to 1.5 times the interquartile range from these percentiles.

229

References

Ament, S. A., Blatti, C. A., Alaux, C., Wheeler, M. M., Toth, A. L., Le Conte, Y., . . . Sinha, S. (2012). New meta-analysis tools reveal common transcriptional regulatory basis for multiple determinants of behavior. Proceedings of the National Academy of Sciences of the United States of America, 109(26), E1801-E1810. doi:10.1073/pnas.1205283109 Ament, S. A., Chan, Q. W., Wheeler, M. M., Nixon, S. E., Johnson, S. P., Rodriguez-Zas, S. L., . . . Robinson, G. E. (2011). Mechanisms of stable lipid loss in a social insect. Journal of Experimental Biology, 214(22), 3808-3821. doi:10.1242/jeb.060244 Brutscher, L. M., McMenamin, A. J., & Flenniken, M. L. (2016). The Buzz about Honey Bee Viruses. PLOS Pathogens, 12(8), e1005757. doi:10.1371/journal.ppat.1005757 de Miranda, J. R., & Genersch, E. Deformed wing virus. (1096-0805 (Electronic)). Farooqui, T., Robinson, K., Vaessin, H., & Smith, B. H. (2003). Modulation of early olfactory processing by an octopaminergic reinforcement pathway in the honeybee. Journal of Neuroscience, 23(12), 5370-5380. Guidugli, K. R., Nascimento, A. M., Amdam, G. V., Barchuk, A. R., Omholt, S., Simoes, Z. L. P., & Hartfelder, K. (2005). Vitellogenin regulates hormonal dynamics in the worker caste of a eusocial insect. Febs Letters, 579(22), 4961-4965. doi:10.1016/j.febslet.2005.07.085 Müßig, L., Richlitzki, A., Rößler, R., Eisenhardt, D., Menzel, R., & Leboulle, G. (2010). Acute Disruption of the NMDA Receptor Subunit NR1 in the Honeybee Brain Selectively Impairs Memory Formation. The Journal of Neuroscience, 30(23), 7817-7825. doi:10.1523/jneurosci.5543-09.2010 Rein, J., Mustard, J. A., Strauch, M., Smith, B. H., & Galizia, C. G. (2013). Octopamine modulates activity of neural networks in the honey bee antennal lobe. Journal of Comparative Physiology. A, Neuroethology, Sensory, Neural, and Behavioral Physiology, 199(11), 947- 962. doi:10.1007/s00359-013-0805-y Ryabov, E. V., Wood, G. R., Fannon, J. M., Moore, J. D., Bull, J. C., Chandler, D., . . . Evans, D. J. (2014). A Virulent Strain of Deformed Wing Virus (DWV) of Honeybees (Apis mellifera) Prevails after Varroa destructor-Mediated, or In Vitro, Transmission. PLOS Pathogens, 10(6), e1004230. doi:10.1371/journal.ppat.1004230 Wang, Y., Brent, C. S., Fennern, E., & Amdam, G. V. (2012). Gustatory perception and fat body energy metabolism are jointly affected by Vitellogenin and juvenile hormone in honey bees. Plos Genetics, 8(6), 13. doi:10.1371/journal.pgen.1002779 Wilfert, L., Long, G., Leggett, H. C., Schmid-Hempel, P., Butlin, R., Martin, S. J., & Boots, M. Deformed wing virus is a recent global epidemic in honeybees driven by Varroa mites.

230

(1095-9203 (Electronic)).

231

Appendix B: Supplementary Tables

Sample 260/230 260/280 Conc. Yield RIN CAGEscan Reads Name (ng/µl) (ng) value

N2 1.48 2.05 31 1540 5.5 14,172,924

N4 0.91 2.07 43 2126 5.3 12,209,666

N29 1.47 2.08 29 1458 4.2 10,652,219

N31 1.87 1.98 35 1746 5.5 10,424,626

N32 0.42 1.88 27 1342 3.8 11,031,651

N33 0.61 2.11 30 1506 4.5 10,998,419

N41 1.70 1.96 37 1826 5.5 12,987,132

N42 0.34 2.06 36 1786 6.7 10,126,459

F14 0.56 1.89 30 1510 6.0 5,449,888

F27 0.77 1.85 34 1690 6.4 3,586,235

F28 0.58 1.69 27 1348 6.9 7,725,895

F41 0.51 2.14 28 1424 7.7 9,237,207

F48 0.57 2.17 36 1778 8.0 2,936,239

F49 0.47 2.28 37 1866 5.8 4,355,366

F50 1.16 2.28 33 1652 7.6 4,307,121

F54 0.45 2.46 34 1710 5.8 2,348,738

Table B.1. A) This table depicts the Nanodrop quality scores, RNA concentration/total quantity, and Bioanalyzer RIN values of each sample used for CAGEscan. Although there is a high degree of variability in RIN score between individual samples, it is unlikely that this reflects RNA degradation that would adversely affect CAGEscan sequencing for the following reasons: 1) The insect 28s ribosomal subunit can dissociate at the temperatures employed by the Bioanalyzer, which can cause spurious RIN values. An examination of the Bioanalyzer electropherograms revealed that this was, in fact, the case (Supplementary Figure S1B). 2) If RIN score was reflective of the RNA quality in these samples, then a positive correlation should be expected to exist between RIN and sequencing/mapping efficiency. However, no such relationship was found (p>0.25, Spearman Rank Correlation). While a modestly significant correlation was found between RIN score and the percentage of

232

CAGE tags successfully mapped (p=0.016), RIN score was inversely (rather than positively) related to CAGE mapping success (r= -0.59).

B) Sample Bioanalyzer Trace. Note the bimodal peak identified as the 18s rRNA (rRNA1), and the lack of a peak corresponding to the 28s rRNA. This indicates that the 28s rRNA dissociated and comigrated with the 18s rRNA. Since calculating the RIN value depends on the proper identification of these rRNAs, this disassociation would result in the RIN giving an inaccurate representation of RNA quality. Still, the electropherogram does not indicate that any RNA degradation occurred (PMID: 21067419, PMID: 24862828).

233

Total Number of Paired- Sample Sample Name NanoCAGE Library end Reads 1 F14 5,449,888 2 F27 3,586,235

3 F28 7,725,895 4 F41 9,237,207 5 F48 Forager 2,936,239 6 F49 4,355,366 7 F50 4,307,121 8 F54 2,348,738 Total = 39,946,689 1 N2 14,172,924 2 N4 12,209,666

3 N29 10,652,219 4 N31 10,424,626 5 N32 Nurse 11,031,651 6 N33 10,998,419 7 N41 12,987,132 8 N42 10,126,459 Total = 92,603,096 Table B.2. Number of reads obtained for each sequenced sample. This table shows general information about the number of reads obtained for each sample in forager and nurse libraries.

234

Total Number of Total Number of Total Number Mapped Reads Mapped Reads Unique Sample Mapping Ratio of Reads (default (estimated Mapped Reads settings) distances)

F14 5,449,888 6,030,292 6,134,426 3,143,064 57.67%

F27 3,586,235 4,002,757 4,071,609 2,157,798 60.16% F28 7,725,895 7,927,736 8,057,070 4,244,582 54.93%

F41 9,237,207 5,895,966 5,981,393 3,130,408 33.88% F48 2,936,239 3,361,068 3,416,792 1,798,240 61.24% F49 4,355,366 5,716,008 5,808,440 3,040,200 69.80%

F50 4,307,121 5,745,833 5,839,990 2,955,469 68.61% F54 2,348,738 3,021,819 3,070,740 1,623,527 69.12%

N2 14,172,924 15,706,801 15,967,039 8,084,168 57.03% N4 12,209,666 13,228,888 13,450,770 7,068,767 57.89% N29 10,652,219 13,739,874 13,981,602 6,949,056 65.23%

N31 10,424,626 12,889,625 13,127,848 6,820,778 65,42% N32 11,031,651 13,393,432 13,621,784 7,173,714 65.02%

N33 10,998,419 14,417,662 14,663,617 7,503,984 68.22% N41 12,987,132 15,571,634 15,821,593 8,050,612 61.98%

N42 10,126,459 12,105,027 12,305,292 6,529,766 64.48%

Table B.3. Mapping statistics of library reads using Tophat. Column 3 shows the number of mapped reads obtained using Tophat when default values for the (average inner distance between mate reads of a pair of reads=20) and (standard deviation of inner distance=50) were used. Column 4 shows the number of mapped reads using Tophat when the estimated values for (average inner distance between mate reads of a pair of reads=588) and (standard deviation of inner distance=767) were used. These values were estimated as follows. We used bowtie2 to map the reads initially. Then, we estimated these values from the distances between mates of the properly mapped paired-end reads. We used these values eventually to map the reads using Tophat. The mapping ratio is shown in column 6.

Note: The values in column 3 may be greater than the original number of reads in column 2. This is because reads are not necessarily paired during the mapping process. Consequently, the number of mapped reads is calculated based on individual mates of paired-reads.

235

Mapped Reads Total Number of Total Number of with High Correctly Within Insertion Unique Sample Paired-End Mapped Reads Mapping Quality Mapped Reads Size (1513)* Mapped Reads Reads F14 5,449,888 6,134,426 5,235,317 4,132,273 3,321,089 1,661,003 F27 3,586,235 4,071,609 3,643,431 2,886,570 2,321,172 1,160,788

F28 7,725,895 8,057,070 7,043,813 5,571,558 4,507,082 2,254,095 F41 9,237,207 5,981,393 5,008,184 3,757,953 3,204,608 1,602,772

F48 2,936,239 3,416,792 3,029,380 2,367,407 1,914,907 957,521

F49 4,355,366 5,808,440 5,155,830 4,120,182 3,290,889 1,645,762 F50 4,307,121 5,839,990 5,001,987 3,976,839 3,151,237 1,575,704

F54 2,348,738 3,070,740 2,741,950 2,152,211 1,717,604 858,840 N2 14,172,924 15,967,039 13,470,127 10,603,418 8,192,48q7 4,096,705 N4 12,209,666 13,450,770 11,815,466 9,345,058 7,329,738 3,665,374

N29 10,652,219 13,981,602 11,670,196 9,301,672 7,308,576 3,654,964

N31 10,424,626 13,127,848 11,581,843 9,301,324 7,174,563 3,587,528

N32 11,031,651 13,621,784 11,967,574 9,366,077 7,295,335 3,648,373

N33 10,998,419 14,663,617 12,600,241 9,984,766 7,710,155 3,855,377

N41 12,987,132 15,821,593 13,401,785 10,486,051 8,014,535 4,007,666

N42 10,126,459 12,305,292 10,788,376 8,302,999 6,482,828 3,241,663

Table B.4. Filtration results on the mapped reads based on quality threshold, mapping both mates of paired- end reads and insertion size. We filtered the raw reads, retaining mapped reads with high mapping quality that were properly paired and with and appropriate distance between the mates of the pair. First three columns are identical to columns 1, 2 and 4, respectively, of Supplementary Table S3.

Column Descriptions:

Column1 (Sample): Sample Name Column2 (Total Number of Paired-End Reads): The original total number of paired end reads Column3 (Total Number of Mapped Reads): The total number of reads mapped by Tophat. The number of mapped reads is calculated based on individual mates of paired-reads because reads are not necessarily paired during the mapping process. For this reason, the numbers in this column may be greater than the original number of reads in column 2. Column4 (Mapped Reads with Good Mapping Quality): The total number of mapped reads which passed the quality threshold (20) Column5 (Correctly Mapped Reads): The total number mapped reads which passed the quality threshold (20) and were properly paired during mapping (an example of the incorrect mapped read is when mates are mapped in the wrong direction) Column6 (Within Insertion Size 1513): The total number mapped reads which passed the quality threshold (20) and were properly paired during mapping and have insertion size <1513 Column 7 (Unique Mapped Reads): The unique number of mapped reads in column 6

236

*1513 is the template length which is: (Average insertion size + standard deviation + paired read lengths) i.e. (588 + 767+ 79*2 ) = 1513

237

% CAGE % CAGE CAGE Tags Tags % CAGE CAGE Tags Tags in (-) Strand Associated Tags Total CAGE in (+) Associated Total CAGE Associated Associated to Genes (-) Sample Tags in (+) Strand to Genes to Tags in (-) to Genes Strand to (Both Strand Associated (+) Strand Strand Strands) to Genes Forager

F14 744,521 636,437 85.48% 616,553 552,489 89.60% 87.35% F27 516,490 448,612 86.85% 429,271 394,422 91.88% 89.14% F28 998,448 812,140 81.34% 795,356 718,421 90.32% 85.32%

F41 704,367 557,640 79.16% 534,496 466,164 87.21% 82.64% F48 425,089 372,402 87.60% 369,030 334,903 90.75% 89.07%

F49 729,469 641,975 88.00% 634,005 578,035 91.17% 89.48% F50 695,749 601,638 86.47% 609,247 553,558 90.85% 88.52%

F54 381,211 330,342 86.65% 336,965 305,528 90.67% 88.54% Nurse N2 1,801,355 1,553,886 86.26% 1,540,413 1,394,063 90.49% 88.22%

N4 1,602,326 1,385,226 86.45% 1,345,999 1,234,192 91.69% 88.84% N29 1,586,950 1,382,406 87.11% 1,362,343 1,249,877 91.74% 89.25%

N31 1,549,518 1,323,682 85.42% 1,280,533 1,190,560 92.97% 88.84% N32 1,597,414 1,399,339 87.60% 1,397,086 1,283,929 91.90% 89.61% N33 1,677,454 1,468,739 87.55% 1,464,781 1,343,610 91.72% 89.50%

N41 1,746,244 1,503,673 86.10% 1,478,731 1,350,915 91.35% 88.52% N42 1,384,485 1,176,963 85.01% 1,200,185 1,100,572 91.70% 88.12% Table B.5. Statistics of the association between CAGE tags and OGS v3.2 genes.

238

Forager Nurse Forager Nurse Upregulated Upregulated Upregulated Upregulated Key Genes Genes TFs TFs Regulators With Sample 534 524 22 4 5 F41 Without 520 660 18 3 5 Sample F41 Percent 77.7% 95.2% 82% 75% 100% Concordance

Table B.6. Comparison of Major Findings with/Without the Outlier F41. Removing the outlier F41 had little impact on subsequent analyses.

239

Nurse HPG vs Nurse Brain Forager HPG vs Forager Brain

Log2 Fold HPG CAGEscan DEG Log2 Fold HPG CAGEscan DEG Change Upregulated Overlap Change Upregulated Overlap Top Genes Top3 Genes

1% >6.5 181 0 1% >7.25 106 5

5% >3.0 591 17 5% >3.0 534 19

10% >2.0 1067 34 10% >2.0 1066 55

20% >1.0 2133 74 20% >1.0 2132 105

Table B.7. Assessing Potential Hypopharyngeal Gland Contamination of Gene Expression. Comparison of nurse and forager HPG upregulated genes with their respective CAGEscan counterparts.

240

Table B.8. Adapters and barcodes used in nanoCAGE libraries.

Adapters ligated to the 5’ end of the CAGE tags:

Strand Sequence (5'-3') 5’ end NNNNNNNNTATA(rG)(rG)(rG)

5’ end barcodes used for nanoCAGE libraries:

Sample Name NanoCAGE Library Barcode

F14 ACAGAT

F27 ATCGTG

F28 CACGAT

F41 Forager CACTGA F48 CTGACG F49 GAGTGA F50 GTATAC F54 TCGAGC N2 ACAGAT

N4 ATCGTG

N29 CACGAT N31 CACTGA Nurse N32 CTGACG N33 GAGTGA N41 GTATAC N42 TCGAGC

241

Table B.9. Nucleotide Sequences for qPCR probes and RNAi constructs used in this study.

242

Appendix C: Supplemental Datasets

Dataset C.1. List of 1,058 differentially expressed genes (DEGs) between nurses and foragers. 534 DEGs were upregulated in foragers and 524 DEGs were upregulated in nurses.

Dataset C.2. Similarity of DEGs between CAGEscan and previous studies.

Dataset C.3. Gene Ontology (GO) analysis based on the category-frequency of enriched GOs (using CateGOrizer).

Dataset C.4. Gene Ontology (GO) analysis based on the frequency of GO terms associated with forager/nurse DEG list relative to genomic background using Fisher’s exact test, followed by FDR correction for multiple testing (FDR < 0.05).

Dataset C.5. List of 26 Transcription Factors were differentially expressed between nurses and foragers, with 4 and 22 upregulated TFs in each context, respectively. These TFs were identified as Key Regulators of Behavioral Maturation.

Dataset C.6. G/C content analysis for the promoters of DEGs.

Dataset C.7. Number of Forager DEG Genes whose promoters were enriched for motifs of TFs.

Dataset C.8. Distance Between nurse and forager TSS (bps) for all 1,058 DEGs.

Dataset C.9. Gene Ontology (GO) analysis of the 646 genes with alternative TSSs using (DAVID GO analysis tool).

Dataset C.10. R Code.

Dataset C.11. Normalized and Imputed Microarray Expression.

Dataset C.12. Microarray ANCOVA Accessory Data.

Dataset C.13. Behavioral Data.

Dataset C.14. qPCR Data

243