Quantitative Immunology for Physicists

GrégoireAltan-Bonnet,1, ∗ Thierry Mora,2, ∗ and Aleksandra M. Walczak2, ∗ 1Immunodynamics Section, Cancer & Inflammation Program, National Cancer Institute, Bethesda MD 20892, USA 2Laboratoire de physique de l’Ecole´ normale supérieure (PSL University), CNRS, Sorbonne Université,Universitéde Paris, 75005 Paris, France The adaptive immune system is a dynamical, self-organized multiscale system that protects vertebrates from both pathogens and internal irregularities, such as tumours. For these reason it fascinates physicists, yet the multitude of different cells, molecules and sub-systems is often also petrifying. Despite this complexity, as experiments on different scales of the adaptive immune system become more quantitative, many physicists have made both theoretical and experimental contributions that help predict the behaviour of ensembles of cells and molecules that participate in an immune response. Here we review some recent contributions with an emphasis on quantitative questions and methodologies. We also provide a more general methods section that presents some of the wide array of theoretical tools used in the field.

CONTENTS A. Cytokine signaling and the JAK-STAT pathway 14 I. Introduction 2 1. Cytokine binding and signaling at equilibrium 15 II. Physical chemistry of ligand-receptor 2. Tunability of cytokine responses. 15 interaction: specificity, sensitivity, kinetics. 4 3. Regulation by cytokine consumption 17 A. Diffusion-limited reaction rates 4 4. Other regulations 17 B. Extrapolating collision rates in solution to B. Communication across space and time, and association rates in the physiological cytokine niches 18 context 5 1. Cytokine dynamics as C. Rates and numbers in the physiological diffusion-degradation 18 context 5 2. Screening by cytokine-consuming cells 18 1. Association rates 5 3. Probability of autocrine capture 19 2. Dissociation rates 5 4. Size of cytokine niches 20 3. Numbers of receptors per cell 5 V. Cell fate 20 4. Typical binding rates and some biology 6 A. Gene regulation and cell differentiation 20 D. Receptor-antigen specificity 7 1. Th1/Th2 differentiation 21 1. Cross-reactivity 7 2. Other differentiation switches 22 2. Models of receptor-antigen binding 7 3. Experimental test of bistability in cell 3. Data-driven receptor-antigen binding differentiation 22 models 8 B. Hematopoiesis 24 4. Modeling immunogeneticity 8 1. Timescales 24 C. Inferring the hematopoiesis differentiation III. Antigen discrimination 9 tree 25 A. T cells 9 D. Cell fate during the immune response 26 1. Kinetic proofreading for ligand 1. Choice and timing of cell fate under discrimination 9

arXiv:1907.03891v2 [q-bio.QM] 28 Jul 2019 stimulation 26 2. Adaptive kinetic proofreading 10 2. An aside on cell types 27 3. Coupling mechanics and biochemistry: 3. Inferring cell fate timelines during the the signiﬁcance of forces for ligand immune response 27 discrimination 11 4. Quorum sensing 29 B. B cells 13 C. Coarse-graining of molecular details and VI. Repertoires 29 model reduction 14 A. Size of immune repertoires 29 B. Inference of the stochastic repertoire IV. Cell-to-cell communication through cytokines 14 generation process 31 C. Thymic selection and central tolerance 32 D. Diversity and the clone size distribution 34 E. Repertoire sharing 35 ∗ Authors are listed alphabetically. F. Optimal receptor distribution 36 2

G. Repertoire response to an immune 12. Population growth rates 62 challenge 37 D. Ecological models 62 1. Generalized Lotka–Volterra models 62 VII. Lymphocyte population dynamics 37 2. Susceptible-Infected-Recovered (SIR) A. Neutral dynamics 37 Models 63 B. A note about “neutral processes” 38 3. Solution of stochastic population C. Population dynamics model with external dynamics with a source 64 signals 39 4. Solution to foward jump process with D. In host HIV dynamics. 41 opposing drift and source 65 5. The Yule process 66 VIII. Aﬃnity maturation 41 E. Inference 66 A. Modeling the hypermutation process 42 1. Probabilistic inference, maximum B. Cycles of selection 42 likelihood and Bayesian statistics 66 C. Evolution of broadly neutralizing 2. Model selection 67 antibodies 43 3. Expectation-Maximization 68 D. Population genetics approaches to aﬃnity 4. Hidden Markov models 68 maturation 44 5. Information theory 69 1. Evolutionary analysis of repertoire 6. Maximum entropy models 69 dynamics 44 7. Machine learning and Neural networks 69 2. Models of co-evolution of phenotypic F. High-throughput repertoire sequencing 69 traits 45

IX. Population dynamics of pathogens and hosts 46 XIII. Acknowledgments 70 A. Viral ﬁtness models 47 References 70 B. Co-evolution between host and pathogen populations 47

X. Discussion 49 I. INTRODUCTION

XI. Glossary 49 The role of the immune system is to detect potential pathogens, confirm they really are undesirable XII. Methods 50 pathogens, and destroy them. The goal is in principle A. Physical kinetics 50 well defined. However recognizing molecular friends from 1. Diffusion-limited reaction rate 50 foes is not easy, and organisms have evolved many com- 2. The rates of dissociation between two plementary ways of dealing with this problem. Immu- biomolecules 51 nologists separate the molecularly non-specific response 3. The formation of ligand-receptor pairs: of the “innate” immune system, which includes every- equilibrium and kinetics 51 thing from scratching to the recognition of protein mo- B. Gene regulation 52 tifs characteristic of bacteria, and the molecularly specific 1. Basic model 52 “adaptive” response, by which specialized cells recognize 2. Auto-amplification with a single evolving features of never encountered before pathogens. transcription factor 52 From another angle we can consider different ways of de- 3. Auto-amplification with multiple stroying a pathogen: either swallowing pathogens whole, transcription factors 52 which is done by cells of the innate immune system called C. Population dynamics, genetics 53 neutrophils and macrophages; or killing our own cells 1. Deterministic mutation-selection that have been infected by a pathogen or are tumourous balance 53 — as performed by representatives of the adaptive im- 2. Genetic drift 54 mune system called killer T-cells. Alternatively, the 3. Wright-Fisher model 54 adaptive immune system produces specialized molecules 4. Probability of extinction 54 called antibodies that smother the invader: they attach 5. Moran model, continuous limit, and to pathogens to prevent them from entering cells and time varying selection 55 multiplying; they bind to bacterial toxins, thereby dis- 6. Branching processes 56 arming them; or they bind directly to bacterial cells, 7. Coalescence process 57 flagging them for consumption by macrophages. 8. Site frequency spectra and tree That short overview gives us an idea of the many balancing 57 strategies that both pathogens and the host organism 9. Clonal interference 58 have at their disposal (Fig. 1). Pathogenic cells (a bac- 10. Quantitative traits 59 teria, virus, or a tumour cell) are programmed to divide. 11. Lineage reconstruction 60 The immune system is there to prevent this. To a large 3 extend, its main challenge is to recognize the unknown. spatial distribution Pathogens are constantly evolving to escape recognition antigen presentation recognition by the immune system. Although the host organism does affinity maturation have a certain list of “warning sign features,” most of repertoire diversity which are taken care of by the innate immune system, it collective sensing is up against a large set of constantly moving targets — as illustrated in our everyday experience by the evolving influenza virus, which requires a new vaccination every year. For this reason, the strategies developed by the immune system are mostly statistical, and require multiple interactions between different types of cells, a lot of signaling checks and balances that leads to a multiplicity of time and length scales (Fig. 2). Despite this complexity, the immune system works remarkably reliably. How do these P pathogen - immune system interactions on many scales dynamically come together co-evolution in a self-organized way to build a complex sensory system against a high dimensional moving target? This review gene regulation breaks this high level question into smaller problems and presents some results and concepts contributed by physi- molecular cell organism population cists. It also presents a summary of the current methods, experimental and theoretical, used in the field. We FIG. 1. The many scales of the immune system. The do not shy away from the biology of the immune sys- immune system works at many scales from the molecular of tem, but to help the physicist navigate the complexity receptor-protein interactions, gene regulation, activation of of immunology, biological details will be introduced as biochemical pathways, to the cellular of cell-to-cell communi- we go along. For an introduction of the immune system cation directly and through signalling molecules, and organ- for the non-specialist, we refer the reader to the short ismal level responses using cells of the innate and adaptive immune systems, to the population level where global viral but excellent book by Lauren Sompayrac [1]. This re- evolution drives the co-evolution of immune systems of differ- view does not aim to be exhaustive, but rather focuses ent individuals. on the important physical concepts of immune function, reducing biological complexity to a minimum whenever appropriate. has one type of receptor; (ii) receptor-antigen binding is From an evolutionary perspective, all organisms have required for cell or receptor proliferation; (iii) offspring of some form of protection. Bacteria protect themselves the stimulated cell have the same receptor as their par- from viruses through specific CRISPR (Clustered Reg- ents; (iv) cells that have receptors that recognize the or- ularly Interspaced Short Palindromic Repeats)-Cas, or ganism’s self molecules (self-antigens) are removed early unspecific restriction modification systems. The innate in their development. The theory was validated by show- immune system is shared by many animals, and is largely ing that B-cells always produce one receptor [6] and later similar between us and flies. The adaptive immune sys- by experiments showing immunological tolerance to fac- tem evolved in jawed vertebrates and also has changed tors introduced in the embryonic period or immediately little between fish and mammals. Plants also have a well- after [7, 8]. Burnet’s theory provides an incredibly suc- studied innate system, and have recently been shown to cessful framework to understand the adaptive immune have elements of adaptive immunity. Immunity is there- system, yet it is not a quantifiable theory that can be fore a basic element of life. However the details of its tested against concrete measurements. While what re- implementation and spatial organisation scale with the mains to be “filled in” may seem like details, these de- organism. tails still provide a lot of puzzles and uncertainties, also Burnet’s clonal selection theory [2], built on previous at the conceptual scale. In this review, we present certain observations and ideas [3–5], provides a theory of the examples of quantified ideas that have recently emerged. adaptive immune system in the same sense that Dar- We also present some (albeit not all) of the quantita- win’s theory provides a theory or framework for evo- tive puzzles, starting from the smallest molecular (sec- lution. Burnet’s theory states that molecules of the tions II and III) and cellular (sections IV and V) scales pathogens stimulate specific B and T lymphocytes (cells and moving to the organismal (sections VI, VII and of the adaptive immune system) among a pre-existing VIII) and population wide (section IX) scales. How do ensemble of possible cells, which leads to proliferation of cells discriminate between self and non-self in such an this specifically selected clone. Molecules thus recognized exquisite way integrating processes taking part on many are called antigens. It explains the diversity and speci- timescales (section III on antigen discrimination)? How ficity of the adaptive immune system, also highlighting do cells communicate and coordinate to orchestrate the its adaptive nature. The framework is often summarized immune response as a collective phenomenon (section IV in four assumptions: (i) each T and B lymphocyte cell on cytokines)? How do cells adopt specialized pheno- 4

ms s min hours daysmonths years time scales This constraint stems from physical-chemical limits on the parameters of these reactions, typically in the pi- ligand-receptor cellular lymphocyte proliferation memory differentiation interactions activation pathogen clearance evolution comolar to millimolar range for the equilibrium dissoci- length scales ation constant KD = koff /kon (we recall that 1nM = −9 −24 −3 10 mol/L = 10 NAvogadroµm ≈ 0.6 molecules per molecule membrane cell lymph node tissue organism population 3 23 −1 µm , with NAvogadro ≈ 6·10 mol ). In this section, we FIG. 2. Placing immune processes in length and discuss the physical considerations leading to estimates timescales. The timescales of immune interactions cary from of the biochemical rates driving in these reactions. Such seconds to years, and the scales from 10−10m for molecular a discussion of the physical chemistry of ligand-receptor interactions to thousand of kilometers for pathogen evolution interactions is necessary to understand key quantitative aspects of immune activation, as well as discrimination between self and non-self antigens by B & T cells (as we types through cell differentiation to span the range of will discuss in Sec. III). functions they must fulfil (section V on cell fate)? How does the system cover the space of possible specificities allowing for complete protection against the unknown (sec- A. Diffusion-limited reaction rates tion VI on repertoires)? How does the immune system as a whole adapt to the changing environment (section VIII) The basic laws of physical kinetics [9] can be used and how does it influence virus evolution (section IX)? to obtain an estimate of the diffusion-limited rate of This review covers a lot of topics, many of them con- molecular association (see Sec. XII A 1 for a derivation): nected. We try to mention these connections, however kon ≤ kdiffusion, with we have tried to make the sections stand alone and the reader does not have to (and probably should not at- kdiffusion = 4π (RLigand + RReceptor)(DLigand + DReceptor) , tempt to) read them all at once, or in the presented or- (2) der. At the end we include a glossary of the biological where RLigand and RReceptor are the radii of the ligand terms (section XI) to help navigate the immunological and of receptor binding pockets (both assumed spheri- terminology. Detailed appendices (Sec. XII) present gen- cal), while DLigand and DReceptor are their diffusion coeffi- eral methods that are used in physical immunology as cients. Since the receptor in Eq. 1 is usually embedded in well as other fields. the lipid bilayer of an immune cell, we can assume it is relatively immobile compared to its ligand L, which diffuses more rapidly in the extracellular medium or in the cyto- II. PHYSICAL CHEMISTRY OF plasm, DReceptor = Dmembrane DLigand = Dsolution. LIGAND-RECEPTOR INTERACTION: The ligand’s diffusion coefficient can be estimated from SPECIFICITY, SENSITIVITY, KINETICS. Stokes-Einstein’s formula:

kBT 2 −1 Our exploration of the immune system starts at the DLigand = Dsolution = ≈ 300 µm s (3) molecular scale, through the binding of ligands and re- 6πηRLigand ceptors expressed at the surface of immune cells. Ligands carry information about the pathogenic threat and can for a small ligand of size RLigand = 1nm, diﬀusing in be of two types: antigens, i.e. bits of proteins that are extracellular medium (whose viscosity is given by that of water, η = 0.7 mP a · s), at body temperature T = recognized by the immune system; and cytokines, which −23 are small molecules secreted by immune cells to commu- 310K with Boltzmann’s constant kB = 1.38 · 10 J/K). nicate with each other about their current state and ex- Then, from Eq. 2, assuming a small target (RReceptor perience. Binding events between these ligands and their RLigand), we obtain: cognate receptors provide the raw signals that cells must 2kBT 9 −1 −1 interpret to adapt their individual and collective behav- kdiﬀusion = 4πRLigandDLigand = ≈ 2 · 10 M s . ior, eventually mounting an immune response in the case 3η (4) of recognition. Eq. 4 gives a general upper bound for any ligand-receptor All immune decisions start with the classical physical association rate, whether it is immunological or not. This chemistry of ligand–receptor interactions: limit is conceptually equivalent to the speed of light (al-

kon though clearly not as fundamental). Yet, there are examples when ligand-receptor associa- Ligand + Receptor Complex −→ Activation (1) koff tions between two small proteins seem to “beat” this diffusion limit: kon > kdiffusion [10]. Such an apparent para- This extremely simple reaction leads to strong limits dox can be resolved when considering the limiting step on how fast signals can spread in the immune system for this association. If the ligand interacts weakly with a (and in biological systems in general), which in turn large object, (e.g. the entire plasma membrane of a cell, allows us to discriminate possible regulatory scenarios. DNA coils), inducing directed diffusion in a constrained 5 space before reaching its specific targets [11, 12], this e6 ∼ 400. Hence, the basal association rate (before pre-equilibration step implies that the cross-subsection taking into account more subtle molecular constraints) of the object our ligand needs to hit can be much larger. is ∼ 5 · 106M −1s−1 rather than the ∼ 2 · 109M −1s−1 The diffusion molecules then can “hop” around these estimated in Eq. 4. Additional corrections should be non-specific binding sites to accelerate their search for made for each pair of biomolecules (as we will discuss the specific binding sites. In these cases, the effective in Sec. II C 4). RReceptor of collision is the macroscopic scale associated with the large object of weak/non-specific interactions (5µm for cells, > 0.1µm for DNA coils), hence, an ap- C. Rates and numbers in the physiological context parent increase in the rate of collision according to Eq. 2. A careful calculation (see e.g. [13]) yields the appropriate 1. Association rates bound. The diffusion coefficient will be slowed down for larger The formula in Eq. 2 is particularly useful in the con- macromolecular complexes, and for molecules diffusing text of quantitative immunology, when one must estimate inside the cells (there, the apparent viscosity can be the kinetics of molecular interactions in different context increased by 10-fold). Additionally, when considering (solution, intracellular cytoplasm, surface plasma mem- biomolecules embedded within cell membranes, one must brane etc.). Most kinetic parameters for biomolecular take into consideration the dramatically increased vis- interactions are measured in solution, as experimental- cosity that leads to slowed-down diffusion for receptor ists routinely purify ligand–receptor pairs and test their proteins on the surface of immune cells (Dsolution ≈ interactions in their soluble form (e.g. by calorimetry or 2 2 200µm /s vs Dmembrane ≈ 5µm /s for a typical protein by surface plasmon resonance – in the latter case, one of radius 2nm). For this reason, when considering a sol- molecule must be immobilized). Such measurements can uble protein interacting with proteins embedded within be used to estimate the hard-to-predict activation bar- the plasma membrane of immune cells (e.g. extracel- rier ∆Gassociation for molecular association by inverting lular cytokines being captured by a cytokine receptor, Eq. 6, using the measured in vitro kon, and the esti- intracellular enzymes –such as kinases or phosphatases, mated kdiffusion. Eq. 2 can be used to rescale the viscosity that catalyze the phosphorylation/dephosphorylation of η and cross-section RLigand + RReceptor of molecular in- proteins– interacting with an activated receptor, etc.), teractions in such a way that one can translate soluble the diffusion of the membrane proteins is so small measurements into physiologically relevant parameters. (Dmembrane Dsolution), than it can be neglected and

kdiﬀusion = 4π (RLigand + RReceptor)(Dsolution) . (5) 2. Dissociation rates

B. Extrapolating collision rates in solution to As discussed below, immunological interactions span −1 association rates in the physiological context the range of very short lived interactions, koff > 10 s or −1 τoff = (koff ) < 0.1 s for antibody binding to its target in the initial phase of an immune response, to extremely While the rate of collision calculated above is an upper long-lived interactions, k < 10−4s−1 or τ > 3 hours bound to the association rate, a very important limita- off off for cytokines interacting with their receptors. These esti- tion must be taken into account: not every molecular mates set a huge range of time scales the immune system collision will lead to their association, and we must es- must deal with, even before taking into consideration de- timate the probability of a successful association event. lays in cellular responses and how these affect the lig- This probability is given by Arrhenius’s law: and environment on time scales ranging from hours to

−∆Gassociation/(kB T ) days. These considerations form the crux of the matter kon = kdiﬀusione , (6) for quantitative immunology at the cellular scale: while where ∆Gassociation is the free energy barrier (entropic the physical chemistry of ligand-receptor interactions is and enthalpic) that molecules must overcome to asso- straightforward, the immune system builds a response of ciate. devilish complexity from such elementary interactions. The estimation of ∆Gassociation is tricky as it requires a deep structural understanding at the atomic level of the entropic, energetic and conformational changes 3. Numbers of receptors per cell associated with bond formation between two large biomolecules. For the entropic contribution, one rule of In some situations, such as in T cell antigen recog- thumb is to estimate the numbers of degrees of freedom nition, where both the T cell receptor and the antigen constrained by the association between biomolecules: exist in a membrane-bound form. All dynamics of inter- simply aligning two biomolecules (in rotation or in trans- actions must be estimated taking into account the surface lation) incurs an entropic cost of at least 6kBT , reduc- concentrations of molecules, with proper adjustment for ing the probability of association of each collision by slower diﬀusion in the association rates. 6

One often measures the number of receptors per cell and trigger a signaling response that activate innate re- #Rcell, e.g. by assessing the number of cytokine recep- sponses. Hence, for the innate immune system, self vs. tors on the surface of cells using quantitative flow cy- non-self discrimination is hard-wired at the molecular tometry, while the ligand is provided in soluble forms, level and gets triggered in the early moments of an im- as is the case with cytokines. In this case, one must mune response. estimate the solution-level concentrations of available re- The adaptive immune system offers a more challenging ceptors [R]total in the reaction volume V : issue. Each B and T cell clone expresses its own unique receptor (and one only), whose assembly is driven by ran- #RcellNcell [R]total = , (7) dom events (described in detail in Sec. VI and Sec. VIII). V NAvogadro The ligands of these random receptors, which are called antigens, are not pre-determined — in fact, they may where N is the number of cells in volume V . For ex- cell not even exist at the time of birth of the organism, if ample, the effective “concentration” of receptors binding for instance they are derived from fast evolving strains the IL-7 cytokine within a lymph node can be estimated, of viruses. Understanding how the binding and unbind- knowing that each of 107 T cells within this 50 µL volume ing rates of such ligand-receptor pairs contribute to self express 103 receptors: vs. non-self discrimination is one of the core issues in [IL − 7R] ≈ 3 · 10−10M = 300pM, (8) quantitative immunology. Here we give some orders of total magnitude of the binding rates to be discriminated. −11 while KD = 10 M = 10pM for IL-7 binding to its re- B and T cell differ fundamentally by the type of ceptor. Hence [IL − 7R]total KD and any secreted IL-7 antigenic ligands they interact with. T-cell receptors will rapidly be captured by the cells within a lymph node. (TCR) interact with their antigen on the surface of The high density of receptors and cells within lymphoid antigen-presenting cells, which they scan for anomalies. tissues makes for an interesting regime of competition These antigens are complexes which include a short pep- for soluble ligands, as discussed below (Sec. IV). Such tide (around 10 amino acid-long) produced within the normalization by the Avogadro number, while straight- antigen-presenting cells by chopping up larger proteins forward, is of critical significance in many immunological expressed by the cell, either from the genome or from configurations, as one must bridge the molecular scale foreign pathogens. Each peptide is loaded onto a large of immune agents (cytokine secretion and consumption) protein called the Major Histocompatibility Complex with the functional scale of the system, where competi- (MHC), forming a peptide-MHC (pMHC) complex. By tion between cells for soluble ligands takes place. contrast, B cell receptors (BCR) bind directly to proteins. The exact position where the binding occurs is called an epitope. Antibodies are soluble versions of the 4. Typical binding rates and some biology BCR with the same antigenic specificity, and also bind directly to the pathogen proteins to neutralize them. We summed up above the basic and general physical The affinity of antibodies (produced by B-cells) are chemistry that drives the molecular association and dis- highly variable. At the onset of an immune responses, B- sociation of molecules in order to delineate key quantita- cells express and secrete Immunoglobulin M (IgM), which tive parameters in immunological regulation. The precise is a pentamerized version of antibodies whose affinity for values of these rates is in fact crucial for the immune sys- the target is weak (KD > 10µM): the ability of secreted tem to recognized pathogens. IgM to bind to pathogens and guide them towards erad- At its core, the immune system can be considered as ication is then driven by the multimeric interaction of a collection of cells, called leukocytes, whose activation IgM with its target. Upon engagement of the adaptive registers the presence of “new” molecules — lipid and immune response, the interplay between T-cell help and nucleic acid signatures of viral and bacterial infection for B-cell somatic hypermutation drives the maturation of the innate system, or unknown proteins and peptides for antibody affinity (see Sec. VIII). At this point, B cells the adaptive immune system — and translates into a increase the affinity of the antibody they express (down defensive response — secretion of neutralizing antibodies, to KD < 1nM) and switch the class of antibodies from killing or phagocytosing of infected targets, etc. IgM to IgG i.e. from a pentameric version to a monomeric In the innate immune system, non-self recognition is version that does not require as much multimerization of encoded in the structure of the ligand and the recep- the antibody to bind to their target. Such decrease in tor. Monocytes, macrophages, dendritic cells, and Mast KD and increase in affinity is not driven by better asso- cells are endowed with genome-encoded receptors called ciation rates: IgM binds to their target with typical kon 6 −1 −1 Toll-Like Receptors whose ligands are non-mutable ele- around 10 M s – similar to IgG. kon is essentially ments of pathogens — peptidoglycans and liposaccha- driven by the collision rate and the contribution of the rides from the membrane of bacteria, CpG unmethy- energetic barrier in the association rate is very limited. lated dinucleotides derived from viruses. Recognition of Instead, the improvement afforded by affinity maturation non-self in that case is a “simple” lock-and-key propo- is driven by a better fit between antibody and antigen, sition whereby pathogenic ligands engage the receptors resulting in a lower koff . 7

In the case of T-cell antigen discrimination, the differ- must be recognized by a large variety of receptors. If ence between ligands that will or not activate the immune there are N antigens, each of which can be bound by k response is extremely sensitive. Some peptides can elicit receptors, and R distinct antigen receptors, each of which a robust activation with a single pMHC molecule, while can recognize ` antigens, then one must have Nk = R`. mutated versions of this peptide differing by just a sin- To fix ideas, the ratio p = `/N = k/R, the probabil- gle amino-acid are unable to trigger T cells even in large ity that a random antigen and receptor bind together, is quantities (> 106). The striking feature in T-cell biology thought to be around 10−5 [17]. is that a single mutation in the peptide of the pMHC It should be emphasized that finding the sequences of only has a marginal effect on koff , while greatly impact- binding pairs of antigens and lymphocyte receptors re- ing its functional capacity to activate T-cells [14, 15]). mains an essentially experimental question, which has Experimentalists can measure the biophysical character- received renewed attention lately thanks to the reduced istics of pMHC–TCR interactions in solution, either by costs of sequencing allowing for high-throughput binding surface–plasmon resonance on soluble pMHC interacting or functional assays [18–21]. However, despite increas- with surface–immobilized TCR, by directly observing the ing amounts of data on these binding pairs compiled in surface of cells using single-molecule imaging and fluores- useful databases [22, 23], and current attempts to lever- cence energy transfer, or by measuring mechanical forces age these data to make prediction using modern machine (see Sec. III A 3). Typically, an agonist ligand, defined learning techniques [24–26], there exists no good predic- as a pMHC that will elicit an immune response, binds tive model of antigen-receptor specificity. Therefore, the to their TCR with τoff ∼ 1 − 10s, while non-agonist lig- binding models presented in the next paragraph should ands bind with timescales that are at least 3-fold shorter be viewed as useful toy models for investigating broad (from 0.3 to 3 s), although this varies according to the properties of cross-reactivity. particular TCR, and also depends on the MHC class (of which there are two, as we will see later). This relatively small difference in binding results in a large fold 2. Models of receptor-antigen binding change (≥ 105) in response, regardless of the concentrations. Understanding how cells can perform such sensi- The recognition process of antigens by immune recep- tive discrimination is a major challenge of quantitative tors is based on molecular interactions between the two immunology, which we will discuss in detail in Sec. III. proteins: the receptor protein and the antigen, which can ∆G(a,r)/kB T be summarized by KD = c0e , where ∆G(a, r) is the interaction free energy of binding between anti- D. Receptor-antigen specificity gen a and receptor r, and c0 a constant. This energy may be modeled as a string matching problem [27]. In T- and B-cell receptors do not interact as set of locks such toy models, each interaction partner is decribed by and keys matched to each other. Instead, each receptor a string of length N representing the interacting amino can bind many different possible ligands, and vice-versa. acids. The binding energy between these two specific in- Here we discuss numbers, data, and models that charac- teraction partners is assumed to depend additively on the terize this many-to-many mapping. interaction between pairs of facing amino-acids:

N X ∆G(a, r) = J (a , r ), (9) 1. Cross-reactivity k k k k=1

Despite the huge diversity of BCR and TCR (reviewed where the interaction matrix, Jk(ak, rk) is an q×q, where in Sec. VI), the diversity of antigens may be even greater. q is the size of the space that defines the variabilty of While such an estimate is difficult for BCR epitopes, a residues at each position. If we describe each residue as rough estimate of the number of antigens can be obtained one of the q = 20 amino acids, we need to define an amino for peptide-MHC complexes. For each of the two MHC acid interaction matrix. The Miyazawa-Jernigan matrix classes that exist, only a few (∼ 5%) percents of pep- [28] was used in such a model to suggest that thymic tides can be presented on any of the 6 MHC genes ex- selection favors moderately interacting residues in TCR pressed in an individual. For peptides of length 12, this [29], or more recently to study the effect of thymic se- amounts to 0.05 × 2012 ≈ 2 · 1014 antigens. While this lection on tumor antigenic peptides [30]. The maximum might be a manageable number for humans, who har- affinity of a receptor to a large set of antigens, which plays bor ∼ 1012 B and T cells, this is not by mice, who have a key role in thymic selection (Sec. VI C), is amenable to ∼ 108 such cells, and this could be an issue even for hu- a statistical mechanis analysis of extreme value statistics mans for longer peptides. This argument led Mason [16] [31, 32], leading to universal features that are robust to to conclude that each TCR and BCR must be able to the details of the models. recognize more than one antigen, a phenomenon called Alternatively, reduced models have been considered, cross-reactivity or polyspecificity. Cross-reactivity has where each residue is defined using a projection that at- a less discussed counterpart, which is that each antigen tempts to capture the main biophysical and biochemical 8 properties in a generalized “shape space” [33]. In the ini- cated models including intra-protein or higher-order in- tial string model [34], ak and rk were bounded natural teractions will be needed to accurately predict receptor- integers, and Jk = ak ⊕rk, where ⊕ is the exlusive OR op- antigen affinity. Ultimately, it would be interesting to erator acting on each digit of the binary representations reconcile the useful picture of an effective binding shape of ak and rk. This choice was motivated by algorithmic space with affinity landscapes inferred from data. A ma- ease rather than biophysical realism. These models were jor roadblock is that most current experimental tech- used by Perelson and collaborators to investigate the ef- niques only allow for varying one element of the receptor- fect of thymic selection [34, 35] or the immune response antigen pair, either testing a library of receptors against [27]. a fixed antigen, or a library of antigens against a fixed A more drastic reduction is to binarize the antigen: for receptor [18]. Full characterization of the binding land- each epitope position, the amino-acid is defined either as scape would require new methods to test double libraries the one present in the viral wild type (ak = 1) or another of antigens and receptors in an ultra-high-throughput one (a mutant, ak = −1) [36]. The selective pressure ex- manner. erted by each receptor, which acts as a “field” on ak, was drawn as a number rk from a random continuous distribution, which reduces the model to Jk(ak, rk) = rkak. In [36], it was additionally assumed that certain posi- 4. Modeling immunogeneticity tions of the viral epitope were constrained to take the wildtype value, ak = 1 for k > M, because of strong con- Rather than modeling the full details of the TCR- servation at these sites. A similar description in terms antigen interaction, an alternative strategy is to focus on of binary strings [37] assumes both the viral epitope and the immunogenic potential of particular antigens, with- BCR to be binary strings (ak = ±1, rk = ±1), with out explicitly modeling the TCR. Such an approach was a fixed interaction strength, Jk(ak, rk) = κkrkak, with followed to predict the response of cancer patients to again conserved viral positions k > M for which ak = 1 immunotherapy. The prediction is based on the knowl- is imposed. Within this description a mismatch between edge of the pMHC antigens presented by the tumor cells, the receptor and the viral “spins” induces an energy called neo-antigens [39]. To estimate the ability of a penalty. The convention is such that smaller energies neoantigen a to be recognized by the TCR repertoire, imply stronger binding and better recognition. Similar a score was calculated to evaluate its similarity to a models have been used recently for co-evolution of BCR database D of antigens known to elicit an immune re- and HIV [36, 37], the results of which we describe in de- sponse: tail in subsection VIII. X S(a) = eksa,a0 , (10) a0∈D 3. Data-driven receptor-antigen binding models where sa,a0 is the alignment score between the neoantigen To go beyond the toy models described above, one a and antigens a0 from the database, and k an adjustable must experimentally map out the binding energies be- parameter. The immunogenicity of a is then predicted tween specific pairs of antigens and receptors. This to be: can be done in a massively parallel way using high- S(a) throughput experiments assaying the binding affinity of I(a) = . (11) many pairs in single experiments. Such an approach was S0 + S(a) applied to a deep mutational scan experiment reporting the dissociation constant, KD, of a large number of anti- This quantity was combined with the likelihood A(a) body variants against a fixed antigenic target, fluorescin that neoantigen a is presented by class-I MHC (predicted [19]. by a neural network model [40]) to form a global score The simplest model assumes an additive contribution −I(a)A(a). With its minus sign, this score reflects the of each residue to the binding free energy E(a, r) = fitness of the tumor cells carrying antigen a. These scores ln(KD/c0) as in (9), but with Jk(ak, rk) = hk(rk) fixed were found to be predictive of patient survival after im- to a single value of the antigen a. However, statistical munotherapy. analysis shows that such an additive model is not able While this similarity-based approach holds great to capture all the variability in the binding energy, ac- promise for generally predicting the immunogeneticity of counting for less than 2/3 of the variability in double and antigens, further tests are needed to validate the method triple mutants. Epistasis, defined as non-additive effects, for broader use. The link between survival and TCR accounts for 25−35% of variability in the binding energy recognition is indirect, and is complicated by the fact between antibodies and the antigen and a large fraction that many neoantigens and tumor clones are involved. of the epistasis was found to be beneficial [38]. Direct functional tests of the immune response against Non-additivity in the binding energy is likely to be a libraries of antigens would help consolidate the founda- general feature of both TCR and BCR. More sophisti- tions of this approach. 9

III. ANTIGEN DISCRIMINATION typically only accumulate a small number of phosphoryl groups, below the threshold NP necessary to acti- We now move to the cellular scale, and describe how vate the immune response, while long-lived complexes the signal propagates in minutes from antigen-receptor can progress more deeply through the phosphorylation binding to biochemical networks, allowing for fine antigen cascade. discrimination. We leave to a later section the discussion Neglecting dephosphorylation (kdeP kP, koff ), the how this signal is later integrated to make decisions about number of pMHC–TCR complexes Ci that accumulate i cell fate and response on timescales spanning hours to phosphorylations scales with i as [41]: days (Sec. V). i kP Ci = C0 . (12) kP + koff A. T cells At steady state, the total number of complexes, Ctotal = PNp 1. Kinetic proofreading for ligand discrimination C0 i=1 Ci, is given by a second-order equation describing the balance between the rate of binding between free antigens and free receptors, kon/(V.NA)(L − Ctotal)(T − To model T-cell antigen discrimination, we remain Ctotal), where L is the total number of antigens, T the within the quantitative parameters of the “lifetime total number of TCR, V the volume, NA –the Avogadro dogma,” in which the lifetime τoff of TCR-pMHC is the number, and the rate of unbinding events, koff Ctotal. In sole determinant of the discrimination process. While the limit when TCR are not limiting, Ctotal T , this this dogma was derived from the biophysical measure- balance yields the total number of complexes [44]: ments on TCR/pMHC interactions [15], as any dogma it must be revisited regularly for exceptions and refine- (kon/(V NA))TL Ctotal ∼ . (13) ments. For example, recent studies in the mechanics of koff + (kon/(V NA))T TCR triggering have highlighted a new mode of signal initiation, as we shall discuss in Sec. III A 3. Further assuming a relatively slow phosphorylation Upon engaging their pMHC ligands, T-cells trigger rate, kP koff , the number of fully phosphorylated com- −NP NP a cascade of phosphorylation events, which consist of plexes CNP scales as ∝ konLkoff = Lkonτoff as a func- adding phosphoryl groups to conserved residues of TCR- tion of the antigen characteristics — concentration L and associated chains. The phosphorylations in turn trigger binding affinity τoff . This scaling shows how alterations other reactions, ultimately activating transcription fac- in the pMHC ligand impacting τoff will get amplified into tors that modulate gene regulatory responses. large differences in the amount of phosphorylation accu- Decision making relative to antigen discrimination can mulating on the TCR: thus be modelled at the phenomenological level by the C (agonist) τ (agonist) Np state of phosphorylation. We assume that, when the Np = off . (14) TCR complex has accumulated a certain number of phos- CNp (non − agonist) τoff (non − agonist) phoryl groups NP, it flips a digital switch (to be specific, the activation of the NFAT or ERK molecules [14]) which In other words, kinetic proofreading “reads off” the defines the onset of T-cell activition. lifetime of the pMHC–TCR complex and amplifies dif- One of the first quantitative models that tackled the ferences in the output. As with the original KPR scheme exquisite specificity of antigen discrimination during T of Hopfield and Ninio, this amplification implies energy cell activation was proposed by McKeithan in 1995 [41], expenditures caused by the phosphorylation steps. Struc- building upon the classical kinetic proofreading (KPR) turally speaking, the TCR complex contains 20 phospho- scheme proposed by Hopfield and Ninio in the context rylation sites (specifically, tyrosine residues), which can of protein translation [42, 43]. Upon binding its lig- potentially participate in the kinetic proofreading cas- and, TCR progresses through the different steps of the cade. This means that NP can be as large as 20, and cascade, controlled by a phosphorylation rate kP and a a two-fold change in pMHC–TCR lifetime could be am- 20 6 de-phosphorylation rate kdeP (Figure 3: A). A key as- plified into a 2 ≈ 10 fold difference in the number pect of the kinetic proofreading is that, at each step, it of phosphorylated receptors CNp . This mechanism can is assumed that the complex is de-phosphorylated into account for the fact that TCR ligands with minute dif- unoccupied receptors upon ligand unbinding: this is a ferences in affinity may elicit very different signals. In reasonable assumption because the CD45 phosphatase addition, it is consistent with the lifetime dogma, in that — the enzyme that removes phosphoryl groups from the τoff has a much more determining impact on activation TCR complex — prevents rebinding of TCR by the anti- than kon. gen due to its large size, thus ensuring complete de- However, KPR is insufficient to capture all aspects phosphorylation before the next binding event. A con- of T cells’ ability to discriminate between structurally- sequence of this resetting upon unbinding is that anti- related ligands. As pointed out by Altan-Bonnet & Ger- gens forming short-lived complexes with the TCR will main [45], T cells not only achieve high specificity in 10

A B Kinetic proofreading (KPR) Adaptive kinetic proofreading (AKPR) 1/t OFF 1/tOFF

1/tOFF 1/tOFF

kON kON kP kP kP kP kP … kP … TCR + pMHC C0 C1 CN TCR + pMHC C0 C1 CN 1/tOFF kdeP kdeP kdeP 1/tOFF kdeP +gS kdeP+gS kdeP+gS

S Response Response

Agonist CN CN (log) (log) Non-agonist

Activation Activation Threshold Threshold #pMHC #pMHC 10-1 100 101 102 103 104 5 10-1 100 101 102 103 104 105 10

Partial discrimination Absolute discrimination

FIG. 3. Biochemical scheme to reconcile specificity, sensitivity and speed in antigen discrimination. A) a classical kinetic proofreading (KPR) scheme amplifies differences in TCR phosphorylation (CN ) based on the lifetime τOFF of the pMHC–TCR complex but gets overwhelmed by a large quantity of pMHC and achieves only partial discrimination of antigens. B) the adaptive kinetic proofreading (AKPR) scheme relies on the activation of a phosphatase (S) to limit spurious activation by a large quantity of non–agonist ligands (see text for details). In the KPR scheme (A) a non-agonist (blue line) can activate a T-cell if it is present in high concentrations. The agonist (red line) activates T-cells also when it is present at lower concentrations. In the AKPR scheme (B) even high non-agonist concentrations cannot lead to T-cell activation, while an agonist still manages to activate T-cells. ligand discrimination, but they also maintain high sen- The resulting adaptive kinetic proofreading (AKPR) sitivity, as they are be able to trigger activation from model [14, 44], which is based on experimental evidence a single agonist ligand [46, 47];. In addition, they represented by Stefanovaˇ et al. [48], quantitatively accounts spond very fast, typically within minutes of encounter- for key features of ligand discrimination by the TCR. ing an antigen presenting cell. These requirements for The adaptation module is introduced through a neg- speed, sensitivity and specificity are hard to achieve all ative feedback mediated by a phosphatase S (which re- at once. For example, a large number of phosphorylation moves phosphoryl groups), which is itself activated by steps in the KPR scheme allows for high discriminability, the engaged TCR in state C1, so that its steady-state but also implies a slower response, as each step must be concentration is S = C1/(C1 + KS), where KS is a slow enough to discriminate between agonists and non- model parameter. The dephosphorylation rate is then agonists. It also affects sensitivity, as more steps imply a enhanced in proportion to the phosphatase concentra- lower chance of making it the activation step. These con- tion, to kdeP + γS. The equations for steady state have siderations demonstrate how quantitative modeling in- a simple closed form, whose solution is given by the root validates a bare KPR scheme, and calls for additional of a fourth-order polynomial. mechanisms. Fig. 3 presents a simple graphical argument illustrating how KPR and AKPR schemes perform in their task of discriminating ligands. In the adaptive kinetic proofread- 2. Adaptive kinetic proofreading ing scheme, the negative feedback mediated by S enforces a higher selectivity of antigen discrimination without af- To fulfill the conflicting requirements of specificity, sen- fecting the high sensitivity of the response. sitivity, and speed, one must expand on the simple kinetic To better understand how this is possible, consider a proofreading scheme by adding a mechanism of adapta- large concentration L of non-agonists. Their unspecific tion to modulate the proofreading steps. engagement with the TCR will cause the first phospho- 11 rylation of the complex into C1, in proportion to their during synapse formation. The model assumes the pres- concentration L. This C1 state activates the phosphatase ence of several receptor-ligand pairs, indexed by i (e.g. S, which in turn increases the specificity of the TCR by the TCR-pMHC pairs is one these pairs). The free energy accelerating its dephosphorylation. When L is large, the F for the membrane system is estimated as: amount of phosphatases, and hence desphosphorylation Z 2 rate, will exactly balance out the amount of non-agonist- X λi h (0)i F = dxdy Ci(x, y) z(x, y) − zi bound TCR complexes, preventing these complexes from 2 i (15) reaching the activation state CN (see Figure 3 B). As Z P 1 2 2 2 a result, for non-agonists C is essentially flat and stays + dxdy γ(∇z(x, y)) + κ(∇ z(x, y)) , N 2 below the threshold for T cell activation, regardless of L.

The activation threshold for CNP must to be set at and the system evolves according to: very low values to account for the extreme sensitivity ∂Ci 2 1 δF of the TCR signalling cascade (a single ligand is suffi- = Di ∇ Ci + ∇ Ci∇ (16) cient to trigger activation). Hence, one must introduce ∂t kBT δCi stochastic equations to appropriately tackle the low num- on off +ki (z)LiRi − ki Ci + ζi, ber of molecules involved in the TCR decision making, although this does not affect the qualitative features of and this particular model. ∂z δF = −M + ζ, (17) To conclude, the adaptive kinetic proofreading scheme ∂t δz satisfies the joint requirements of ligand discrimination and sensitivity. It can explain how a single agonist lig- where z(x, y) is the membrane coordinate for the T-cell and can trigger T cell activation, while a large number membrane, Ri, Li, and Ci, are the surface densities of (> 105) of non-agonist ligands cannot. Additionally, it receptors, ligands, and receptor-ligand complexes of the th (0) can account for the speed of ligand discrimination by T i type, zi is the length of the Ci = Ri-Li complex leukocytes, as only two steps can be sufficient to sort lig- bond at rest; and the ζ’s are thermal noises for each ands [49]. It is important to emphasize again the impor- process (their distributions are not explicitely stated in tance of a quantitative approach and of modeling to ap- the original publications [50]). The free energy F sums preciate why achieving speed, sensitivity and specificity up the elastic energies of each type of complex i (with on in ligand discrimination is indeed such an amazing feat stiffness λi), and the elastic energy of the membrane. ki off of the immune system. and ki are the binding and unbinding rates of ligand- receptor complex formation. This model is purely mechanical and passive, in the 3. Coupling mechanics and biochemistry: the significance sense that no energy is injected nor is there any bio- of forces for ligand discrimination chemical feedback. Yet it captures salient features of the spontaneous formation of immunological synapses. Importantly, synapse formation strongly depends on the One of the most striking events in the T-cell activa- lifetime of ligand-receptor complexes, and thus allows for tion process is the dynamic membrane reorganization fine discrimination between agonist and non-agonist lig- into what has been called an immunological synapse. ands. The model also emphasizes the functional relevance of size differences in ligand-receptor pairs. For in- Within minutes of signal initiation, the antigen recep- (0) tors on the surface of lymphocytes congregate at the stance, TCR-pMHC complexes have zi = 15 nm, while center of the interface with their antigen-presenting cells adhesion complexes such as ICAM-1/LFA-1 are much (0) (this has been dubbed a central Supramolecular Assem- larger, zi = 42 nm, as it was confirmed experimentally bly Complex or c-SMAC), while transmembrane recep- [52, 53]. tors and phosphatases needed for signal propagation ac- This model makes several predictions that were vali- cumulate at the periphery of the contact area (p-SMAC dated experimentally. First, it explains how the forma- and d-SMAC respectively, standing for peripheral and tion of the characteristic bullseye pattern emerges from distal). Such a bullseye pattern has been studied quanti- the passive sorting of molecules based on their size, upon tatively using high-resolution and single-cell time-lapsed coupling with the mechanical deformation of the surface microscopy. The functional significance of immunological membrane. Second, the model predicts an optimum for synapses has been identified for the long-term activation the lifetime of the pMHC-TCR complex for which the of T cells, such as directed killing of antigen-presenting accumulation of antigen in the center of the synapse is cells or directed secretion of cytokines [51]. As the cell maximal. For weak antigens, the membrane energy is biology of immunological synapse formation was being dominated by the binding of adhesive complexes, and studied, the question arose as to whether such cellular few antigen-TCR bonds can form because of molecular structure could contribute to self vs non–self discrimina- crowding. For very strong antigens, at intermediate time tion. Qi et al. [50] introduced a Ginzburg-Landau model points (≈ 5min) or with ligands of intermediate affinity, for the mechanical deformation of the T-cell membrane the membranes of the T-cells adopt an inverted bullseye 12

Antigen-presenting cell pMHC 42nm < 2min 15nm LFA-1 ICAM-1 TCR T cell Inverted z synapse y x > 5min with strong antigens 42nm 15nm

Immunological pSMAC cSMAC pSMAC synapse

FIG. 4. Molecular sorting in the immunological synapse. Upon activation by antigen-presenting cells, T-cells reorganize their membrane proteins to form a bullseye structure (so–called immunological synapse) that stabilizes cell-cell interactions and drives cellular functions. Coupling of mechanical forces derived from the curvature of the membrane and differences in the sizes of pMHC/TCR and integrin (LFA-1/ICAM-1) complexes drives the formation of two ”Supramolecular Assembly Complexes” (central: cSMAC; peripheral: pSMAC). At early time points (¡ 2min) or for weaker antigens, the systems form an inverted synapse with the tall, rapidly-formed integrin bonds concentrating in the center (red bonds), and pushing the slowly-formed pMHC-TCR bonds to the periphery (blue bonds). For longer timescales and when strong antigen ligands are present, these molecular complexes get sorted into a classical immunological synapse to alleviate the cost of negative curvature in the inverted synapse. A Ginzburg–Landau model (as introduced by Qi et al. [50]) demonstrates that such self-organized sorting could help in antigen discrimination. pattern with TCR being in the p-SMAC and integrins presence of many non-agonist ligands. More recently, its being in the c-SMAC. Such patterns can be explained interest has been rekindled by new observations empha- when the binding affinity of the integrin binding is larger sizing the importance of mechanics for ligand discrimina- than the binding energy of the weak ligands. In this tion. case the integrins populate the center of the c-SMAC Using a biophysical setup to measure forces and life- and make the deformation of the membrane ”affordable” times of individual pMHC-TCR pairs under tension, Zhu energetically (see Figure 4). While this “bell-shaped” and colleagues [61, 62] discovered two types of bonds de- activation as a function of antigen affinity has been doc- pending on the peptide sequence: slip bonds and catch umented experimentally [54, 55], it has not been consis- bonds. The lifetime of slip bonds decreases monotonously tently observed and there is a sufficient number of ex- under tension: in other words, they break more easily ceptions [56, 57] to diminish its significance, especially at when pulled. By contrast, the lifetime of catch bonds in- the time when clinicians are engineering potent chimeric creases under tension. Molecular dynamics simulations antigen receptors with unphysiologically–large affinity for [63, 64] suggest that the bonds tension helps expose new their ligands [58, 59]. Third, the state of the membranes residues in pMHC, inducing a better contact with the of both the T cell and antigen-presenting cell is expected TCR. At high forces, catch bonds do ultimately break as to affect the synapse and thus T-cell activation, as was well, so there exists an optimal force of around 10 pN for observed in experiments where the cytoskeleton has been which the catch bond lifetime is longest. depolymerized [60]. This distinction between slip and catch bonds has been On the other hand, the model also fails to explain suggested to have functional significance in terms of T- more recent observations, such as the ability of T cells cell activation. For instance, it was shown [63] that TCR- to respond to a single agonist ligand while ignoring the pMHC pairs with similar binding constants elicited dif- 13 ferent activation modes in T cells, with the stimulatory tivation is appealing because it is consistent with the for- pair forming a catch-bond and the non-stimulatory a slip mation of microclusters of BCR on the surface of B cells bond. in response to antigens, as observed by super-resolution How these catch bonds relate to signalling remains microscopy. to be explained in detail, but this new effect may be An alternative model proposed by Reth and col- critical for predicting binding pairs, and immunogenic leagues [70] assumes that receptors cluster even in the neo-antigens of crucial relevance to immunotherapies (see absence of antigens. When in clusters, BCR inhibit Sec. II D). However, the ability of pMHC to activate the each other and do not signal, but they can assemble TCR signalling pathway might still be set by the lifetime and disassemble dynamically. Monomers coming out of of the pMHC-TCR complex, but with a “catch”: this these clusters are more prone to signaling, and antigen- lifetime would have to be assessed under the tension ex- binding stabilizes this monomeric state supposedly be- erted by the membrane dynamics, e.g. 10 pN. This would cause of steric hindrances preventing the return into the also explain the dependence of activation upon mem- clustered/inhibited state. This “dissocation-activation” brane properties such as stiffness. The Ginzburg-Laudau model explains how B cells limit spurious activation of model of Eq. 15 will need to be revisited to account for its ∼ 120, 000 surface BCR through their clustering, with off the existence of catch bonds, e.g. by letting the ki of signaling made only possible by isolated, antigen-bound (0) pMHC-TCR pairs depend on the tension λi(z − zi ), monomers. The measurements of Reth and colleagues possibly affecting how ligands get sorted by mechanical pose a theoretical challenge in terms of understanding forces in the immunological synapse. the role dynamic clustering-release-re-clustering of BCR Additional measurements in recent years (in partic- during B cell activation. A full theoretical model of such ular, using super–resolution microscopy [65]) emphasize process and its impact on antigen discrimination remains the active role that cytoskeletal rearrangements may play to be proposed. in concentrating TCR in the synapse [66] and the com- One observation that must be taken into account when plex interplay between membrane ruffling, receptor sort- considering B cell activation is the role of membrane ing, and mechanical tensions. A more complete quanti- spreading and contraction. Upon initiation of the signal- tative model would help identify key limiting steps de- ing cascade, B cells rapidly (< 10 min) reorganize their ciding how strongly T cells get activated in different con- cytoskeleton and membrane, first by spreading to cap- texts — different antigen-presenting cells, or different co- ture as many antigens on the presenting cells as possible, stimulatory contexts. second by contracting to “concentrate” the active receptors. The quantitative model proposed in [71] accounts for binding and unbinding events driving cell spreading B. B cells and contraction. The model predicts that such a dynamic process can increase dramatically the number of agonist As mentioned before, the affinity of B-cells with their ligands that get captured, compared to a static interface, cognate antigen undergoes a Darwinian selection process enhancing the difference between weak and strong lig- that improves their affinities from KD = 10 µM down to ands, while weaker affinity antigens fail to concentrate. 100pM. Simple counting of the number of occupied recep- A critical aspect of the model is that, if B cells fail to trig- tors in equilibrium could be sufficient to enforce antigen ger a sufficient number of BCR by 1 min (a hard cut-off), discrimination in B-cells. As for TCR binding to pMHC, they terminate the process and shut down their signaling the association rate of BCR to antigens is constant for response. Alternatively, the activation response switches all antigens. But unlike TCR-pMHC binding, which is to a contraction phase with the surface area decaying −0.35 subjected to an activation barrier, this rate is essentially according to the phenomenological law Amaxt with 6 −1 −1 diffusion limited, kon > 10 M s . Strong and weak Amax being the largest area the B cell spreads to, t is antigens thus only differ by their binding lifetimes τoff , the time, and 0.35 is an experimentally-determined ex- which varies between 0.1s and 104 s, suggesting a possi- ponent. This model is phenomenological as it does not ble kinetic proofreading scheme as for TCR [67]. How- model explicitly the biochemical mechanism driving the ever, since the number of potential phosphorylation sites spreading and contraction, and makes ad hoc assump- in the BCR complex is small compared to TCR, such a tions about their behaviour. Yet it illustrates quantita- mechanism may not be as important. tively how such membrane dynamics can help discrimi- The previously mentioned monomeric version of the nate ligands. BCR, IgG, is itself actually a dimer (so that IgM is a To conclude, although the issue of antigen discrimina- pentamer of dimers), with two binding sites. By anal- tion in B cells may not be as stringent as for T cells, ogy with other dimeric receptors on the surface of cells it poses interesting quantitative issues for a different pa- (e.g. Epidermal Growth Factor Receptors or Insulin Re- rameter range of binding constants (KD = 100 pM - 10 ceptors [68, 69]), one could assume that this dimerization µM), over longer timescales (>10min), and using differ- would help concentrate kinases (the enzymes responsible ent cellular mechanisms: receptor clustering and mem- for phosphorylation) around the receptors, causing them brane dynamics. It is worth recalling that this initial to form clusters. This “cross-linking model” of BCR ac- recognition process connects to additional processes over 14 longer timescales, such as validation by helper T-cells, pled differential equations highlighting its main features and dynamics within germinal centers. We will come of adaptation and discrimination, and reveals the broad back to these questions in Sec. VIII. design principles that implement these features. Beyond this particular example, this kind of approach has great potential for helping to make sense of complex C. Coarse-graining of molecular details and model biological systems with many entities and interactions, reduction as is often the case in the immune system.

As we go deeper into the molecular details of immune recognition, the number of molecular species, reactions, IV. CELL-TO-CELL COMMUNICATION intermediates, and therefore parameters explodes. This THROUGH CYTOKINES poses a challenge for a number of reasons. First, the effective behaviour of the system as a whole may still be Cytokine communication is critical to synchronizing relatively simple, suggesting that simpler phenomenolog- the activation of various immune cells and to bridging ical models may describe these processes equally well. multiple spatial-temporal scales in immune responses, While the variables of such models may be hard to relate from local or individual cell activation to global, sys- directly to molecular entities, they are easier to inter- temic responses. Cytokines are small glycoproteins that pret and allow for better analytical progress and predic- get produced and secreted by all cells, immune or not, tions. Second, even assuming that the full complexity of with varied dynamics, amplitude and frequency. These all interactions is needed, large numbers of parameters cytokines then diffuse and bind to receptors present on are likely to lead to overfitting problems, meaning that the surface of adjacent cells to elicit a signaling response many parameters or combinations of parameters are un- that trigger a gene regulatory response. In short, cells derdetermined, undermining the accuracy of predictions. use cytokines to communicate between themselves. In And even when they can be determined, it is not always this section, we discuss quantitative models that have clear which ones need to be fine-tuned to ensure proper been introduced to model how individual leukocytes re- function, or what are the broad design principles presid- spond to cytokines (e.g. using the JAK-STAT pathway) ing over their choices. In Sec. XII E 2, we briefly review over multiple time- and length-scales. the principles of model selection – the classical approach of reducing model complexity from statistics. However, that approach requires to have first defined a hierarchy A. Cytokine signaling and the JAK-STAT pathway of models to test, from least to most complex. Besides, model selection relies on goodness of fit as a criterion to Cytokine receptors are heterodimers, or more rarely evaluate models, while in many cases we may be more in- heterotrimers, so that they carry at least two intracellu- terested in capturing the principal features of a biological lar signaling domains. The JAK-STAT pathway is the function, rather than fitting all the data. dominant pathway engaged by cytokine signaling. Upon In another approach to reducing complexity, Fran¸cois binding by its cognate cytokine, the receptor undergoes a and Hakim [72] developed an method for generating sim- conformational change that presents phosphoryl attach- ple molecular networks in silico that realize a desired bio- ment sites on the JAK (Janus Kinase) receptor to face logical function, simply based on a genetic algorithm that the kinase domain that induces phosphorylation, which selects the “best” solutions. Applied to the problem of in turn leads to activation. Then a protein called STAT absolute ligand discrimination reviewed in Sec. III A, this (Signal Transducer and Activator of Transcription) in- method infers a class of network motifs, called “adaptive teracts with the intracellular domains of the receptors, sorting”, that recapitulates known features of T cell ac- gets phosphorylated by the activated JAK, and dimer- tivation [49]. In particular, it predicts the emergence of izes upon release from the receptor. Dimers of STAT kinetic proofreading and biochemical adaptation. How- then translocate into the cell nucleus, bind to the chro- ever, the chemical species and reactions of the networks matin in specific sites and elicit a transcriptional re- produced by that method may generally not be directly sponse. Such signal transduction is one of the simplest related to the known actors of the phenomenon under biological pathway connecting the extracellular environ- study. ment and its messenger cytokines to a transcriptional re- To reduce model complexity while keeping close to the sponse (Figure 5). Its complexity is encoded in the large details of actual biochemical reactions, one can instead number of pairs of cytokines (> 40) and their receptors start from a complex biochemical network described by that cells can express, as well as the 7 forms of STAT many ordinary differential equations, and “prune” its pa- and 72 = 49 homo- or hetero-dimers that they can form rameters by setting them, individually or by their combi- upon activation [75]. Additionally, the dynamics of cy- nations (ratio or products), to 0 or infinity [73]. Apply- tokine signaling enriches the biology of these pathways, ing this strategy to a complex model of T cell recogni- with the existence of positive and negative feedback reg- tion [74], which contains close to 100 parameters, shows ulations: cytokine signaling inducing the expression of that its behaviour can be boiled down to just three cou- additional cytokine receptors, cytokine degradation, cy- 15 tokine receptor endocytosis, expression of negative regu- the dissociation constant of STAT to the receptor, which lators such as Suppressor of Cytokine Signaling (SOCS). controls the specificity between different receptors and Here we present simple derivations that can help under- different variants of STAT. Note that, for most signal- stand the quantitative regulation of cytokine communi- ing networks, the identity of phosphatases that take care cation within the immune system, which underlies the of dephosphorylating receptors and kinases remain often coordination and orchestration of the immune response. undetermined (because of overlapping and pleiotropic ac- tivities): their activity is modeled phenomenologically as a single rate reaction. 1. Cytokine binding and signaling at equilibrium This system reaches steady-state within minutes [75, 76] such that one can solve d[pSTAT]/dt in Eq. 20 In this section, we discuss the contribution of the for pSTAT (with the conservation of matter condition field of quantitative immunology to understanding the [pSTAT]+[STAT]=Stotal) as a quadratic equation. Com- JAK-STAT pathway as a signal transduction cascade in bined with Eq. 19, the result gives a direct expression of well-mixed conditions. Spatial heterogeneities in cell-to- the response STAT as a function of the input cytokine cell communication via cytokines will be tackled in sec- concentration. This model serves as the basic building tion IV B. block to tackle the dynamic complexity of cytokine re- Within an individual cell, the biochemical reactions of sponses. the JAK-STAT pathway can be treated as well-mixed: the typical concentrations of molecules are fairly high, with ∼ 1000 molecules per cell of diameter 10µm, which 2. Tunability of cytokine responses. translates into concentrations ∼ 100nM. We also assume that most reactions are essentially diffusion-limited, as In the previous section we assumed that cytokines, 7 −1 −1 discussed before: kon > 10 M s . The signal trans- which are formed of several sub-units, were in a pre- duction cascade for JAK-STAT cascade can generically assembled configuration. When that is not the case, cy- be modelled in a step-wise manner, as sketched in Fig- tokine binding to the cytokine receptor can be modeled ure 5. as a two-step scheme. The kinetics of cytokine binding to its receptor action on vating JAK into JAK∗ can be modeled as a simple bi- k1 k2 L + R1 + R2 LR1 + R2 LR1R2. (21) molecular process: off off k1 k2 d #Receptor∗ = k [Cytokine]#Receptor−k #Receptor∗, The cytokine ligand L binds to the cytokine receptor dt on off (18) composed of two parts R1 and R2. A simple equilib- ∗ rium model can be solved, with the dissociation constants where #Receptor is the number of cytokine-engaged re- off on ceptors, [Cytokine] is the cytokine concentration. Be- Ki = ki /ki : cause binding is very strong (typically KD = koff /kon ≈ K1[LR1] = [L][R1] = [L] ([R1]total − [LR1]) 10pM with k−1 ≈ 3600 s and k ≈ 3·107M −1s−1), most off on K [LR R ] = [LR ][R ] = [LR ] ([R ] − [LR R ]) cytokine molecules bind at a diffusion-limited speed with 2 1 2 1 2 1 2 total 1 2 very strong binding. We assume a large extracellular that yields the total concentration of fully-engaged and volume, so that cells do not deplete cytokines as they signaling receptors: bind them. Receptors are usually pre-loaded with JAK, waiting for the conformational change associated with cy- [R2]total [LR1R2] = . (22) tokine binding, to induce phosphorylation. From Eq. 18 K2 K1 1 + [R ] 1 + [L] the number of activated JAK (JAK∗) at steady-state is: 1 total This two-step model was shown to be valid for one ∗ [Cytokine] of the most studied cytokines, IL-2 [77]. IL-2 binds JAK = JAKtotal , (19) KD + [Cytokine] weakly to the abundant α chain of the IL-2 receptor (R1 = IL2Rα), before being “locked” into a stable and where JAKtotal is the total JAK receptor concentration. signaling complex by association with the two other The kinetics of STAT phosphorylation into pSTAT is de- chains (R2 = IL2Rβ-γC ) to form the full IL-2−IL2R com- scribed using a classical Michaelis-Menten equation: plex. In that case, K1[L] 1 such that equation 22 simplifies to: d ∗ [STAT] [pSTAT] = kcat[JAK ] −kdephos[pSTAT], dt [STAT] + Km [L] [LR1R2] = [R2]total . (23) (20) [L] + K1K2 ∗ ∗ [R1]total where [JAK ] = JAK /(V NA) is the concentration of activated JAK within the cytoplasmic volume V , pSTAT This equation reveals key insights for IL-2 and other cy- is the phosphorylated form of STAT, kcat and kdephos are tokines [78]: the amplitude of cytokine signaling is pro- phosphorylation and dephosphorylation rates, and Km is portional the number of β − γ part of the IL2 receptor 16

1 2 3 4 6 Cytokine JAK STAT STAT dimer STAT dimer translocation binding activation phosphorylation release & receptor endocytosis/degradation

Cytokine

JAK JAK JAK JAK JAK JAK JAK JAK Receptor recycling P P Endocytosis STAT STAT & degradation P P P P P P P cytokine phosphorylations STAT STAT receptor P nuclear translocation q 8 P Feedback regulation 7 Gene regulation STAT STAT for cytokine & cytokine receptor P

FIG. 5. Model of the regulation of JAK-STAT signaling in response to cytokines. This model in the adiabatic regime can be solved analytically.

(the limiting part), and reaches half of its maximum at the other cells of this key cytokine for anti-apoptosis and [L]50 = K1K2/[R1]total, inversely proportional to the α proliferation in the T-cell compartment. Quantitative chain of the IL-2 receptor (the non limiting-part). models of such competition for IL-2 based on differential Such tunability of both the amplitude and sensitivity expression of IL-2Rα receptors have illuminated how self of the dose response of cytokines based on the number vs. non-self discrimination can emerge from such IL-2 of cytokine receptor chains can be critical to achieve im- tug–of–war [77, 79, 80]. mune plasticity, as demonstrated in the context of com- Some cytokine receptors have an extracellular domain petition for limited amounts of cytokines. For example, that gets secreted extracellularly, in a soluble form. the tug-of-war for IL-2 between effector T-cells and regu- These can “pre-bind” the cytokine in the extracellu- latory T-cells is mediated by the exact levels of IL2Rα on lar milieu, and deliver it to the complementary chain the surface of these cells. Stimulated effector T-cells initi- bound to the cell membrane: such molecular event can ate an immune response, whereas stimulated regulatory T have positive or negative effects on cytokine signaling cells (Treg) downregulate the response to suppress auto- depending on the context. If the pre–formed soluble immune responses (effector response to self-antigens). In cytokine/receptor complex binds to the cell to form an general, Treg cells are thought to bypass negative thymic incomplete receptor which lacks the intracellular signal- selection despite their strong recognition of self-antigens. ing domain of the soluble cytokine receptor, the JAK These cells then act as pre-activated sentinels that re- misses its trans-phosphorylation partner and the cy- spond to inflammatory cues (such as the upregulation tokine/cytokine receptor fails to signal— “no clap from of self-antigens) while not contributing to inflammation one hand.” Such soluble complexes act as decoys that itself through cytokine secretion: Tregs constitutively ex- antagonize cytokine response (as with viral analogues of press transcription factor FoxP3, a general downregulator IFN receptors, or IL-1 receptors), limiting inflammation of cytokine production. to the benefit of viruses [81]. Whenever an effector T-cell initiates a spurious re- Alternatively, secretion of soluble cytokine receptors sponse to self-antigens, Tregs can extinguish it by con- can trans-activate cytokine signaling and result in a boost suming cytokines (e.g. IL-2) and by downregulating their in cytokine response (as with IL-1, IL-2 or IL-6). For inflammatory impact. In this tug of war, whichever example, in the case of IL-6, the soluble portion of the cell type, effector or regulatory, expresses more receptors IL-6 receptor (sIL-6R) gets secreted to stabilize IL-6 in boosts its ability to capture the cytokine and deprives the extracellular medium (most cytokines are very small 17 proteins with short half-lives) and to accelerate the as- One functional consequence is that cytokines get sembly of a complete IL-6/IL-6R/gp30 signaling complex rapidly consumed while triggering signaling. At the more by binding to gp130 dimers on the surface of cells: such global level, one can integrate a dynamic equation for activation of the IL-6 signaling response thus does not reproduction/consumption that accounts for the rise and quire the presence of the IL-6 receptor in the membrane decay of inflammatory signals in the immune system: of the receiving cells, but only the ubiquitous presence of gp130: this is called cytokine trans-activation [82]. d [Cytokine] = κprod − κconsum. (24) Hence, depending on the exact molecular details, se- dt cretion of cytokine receptors in the extracellular envi- In the most simple case, κ = N k /(V N ) is ronment can trigger or antagonize the cytokine signaling prod prod prod A fixed by the number Nprod of activated cells at a given response. The role of quantitative immunology in that time, the rate of secretion k per individual cells, and context is then to tease out the physico-chemical param- prod the extracellular volume V . The rate κconsum is deter- eters of such regulation to better understand in which mined by the rate of binding of the cytokine to its recep- regime inflammatory cues are regulated. tor on the surface of the cells: κconsum = konNconsumNR[Cytokine]/V NA, where Nconsum is the number of consuming cells and 3. Regulation by cytokine consumption NR the number of receptors per cell. The time dependency of Nprod, Nconsum, κprod, and NR can be arbitrar- Immune responses need tight regulation to avoid spuri- ily complicated and must be parametrized for each im- ous auto-immune activation. This is particularly impor- munological setting under consideration. For example, tant for cytokine regulation as overabundant cytokines in the case of IL-2 in the early events of an immune re- can be extremely deleterious to the organism as a whole. sponse within a lymph node, V ≈ 50µ` (free volume), −1 7 −1 −1 For example, high concentrations of cytokines induce a kprod = 10s , kon = 3 · 10 M s , Nconsum = 10, 000 “capillary leak syndrome,” whereby tissues lose their bar- Treg cells, each endowed with typically NR = 3, 000 rier against blood serum: this results in septic shocks or receptors. For IL-2 to accumulate and reach a signifi- viral hemorrhagic fevers. One simple mode of regula- cant concentration (typically, 10−11M to trigger STAT5 tion of cytokine signaling by the JAK-STAT pathway is phosphorylation), one must have κprod > κconsum, i.e. simply to limit the availability of cytokines in the extra- Nprod > 1, 000. This estimate illustrates that there exist cellular medium. Most cytokines are very small proteins thresholds of activation for immune responses, whereby (with molecular weights in the 10-20kDa range) such that the systems needs a critical mass of activated, cytokine- their half-life in the bodily fluid is short. For example, secreting cells to overcome consumption and drive acti- the half-life of IL-2 injected intravenously for the treat- vation and differentiation [86]. The estimate above does ment of renal cell carcinoma is at most 3 h in the serum. not take into account some of the intricacies of IL-2 reg- More significantly, even in the extracellular environment ulation: positive feedback in IL-2 secretion, recycling of of lymphoid organs, cytokines have a very short half-life IL2Rα chains, upregulation of IL-2R in Treg cells have due to rapid binding and consumption. been documented. To account for these, Eq. 24 must be Given the strong binding of cytokine to their receptors solved numerically in more complex settings [84, 85]. −1 (typically KD < 100 pM or koff < (3000s) ), immune cells can rely on binding to switch off the cytokine signaling response through buffering. This resetting process 4. Other regulations relies on cytokine consumption by endocytosis of the cytokine with its receptor, trafficking them towards lyso- Cytokine consumption is one major mechanism to limit somes, release of the cytokine because of low pH within the duration of availability of cytokines in the extracel- the lysosomal compartment, and degradation. Note that lular medium, but there exist additional, cell-intrinsic the signal transduction may persist after receptor endo- mechanisms that also limit or expand the duration of cytosis as long as the cytokine-receptor pair remains en- JAK-STAT signaling in cells, as illustrated by the fol- gaged [83]. What happens to the cytokine receptor in lowing examples. this process varies based on the cytokine identity. For Certain cytokine pathways rely on intracellularly example, the endocytosed IL-2 receptor dissociates and stored pools of receptors that get recruited upon recep- has its chains sorted towards different compartments: the tor engagement and JAK-STAT signaling. This recruit- IL2RγC chain goes to the lysosome and gets degraded, ment further fuels JAK-STAT signaling by avoiding sat- while the IL2Rα chain goes to early endosomes and gets uration of the initial receptors on the membrane surface. recycled. This process of endocytosis, sorting, recycling This process was analyzed quantitatively in Ref. [87] to and degradation can be very complex at the molecular reveal how a certain type of leukocyte called erythroid level. Yet, a coarse-grained model of this process as a progenitors can sense a cytokine called Epo over a large single-biochemical step can be sufficient when modeling range of concentrations (> 1000-fold range), where classi- IL-2 availability [84, 85]: the typical rate for this step has cal ligand-receptor binding would predict fast saturation been measured to be of the order of (900 s)−1. with increasing cytokine concentration. 18

Another positive feedback in JAK-STAT signaling ex- and time, over the relatively short timescales (minute to ists in the IL-2 signaling pathway: The α chain of the hours) over which cytokines convey information between IL-2 receptor gets upregulated upon IL-2 signaling, low- cells. ering the cytokine concentration at which activation is half-maximum (by virtue of Eq. 23), hence driving further IL-2 signaling [77–79]. Such positive feedback was 1. Cytokine dynamics as diffusion-degradation shown to be critical to commiting cells to long-term JAK activation and proliferation [85]. The propagation of cytokines within tissues is governed Alternatively, JAK-STAT pathways are also endowed by reaction-diffusion equations which generalize Eq. 24: with negative regulators, such as SOCS and Cytokine In- ducible SH2-containing (CISH) proteins, which compete ∂c(~r, t) 2 = D∇ c(~r, t) + κprod(~r, t) − κconsum(~r, t), (25) with STAT for JAK binding, preventing activation [88]. ∂t This can be quantitatively modeled with SOCS or CISH where c(~r, t) is the spatio-temporal profile of the concen- binding competitively with a stronger Km than STAT tration of cytokine, D is the diffusion coefficient for the onto the receptor (Eq. 20). Varying the exact value of cytokine in the extracellular medium, κprod and κcons are Km allows SOCS and CISH to negatively regulate JAK volumic rates of cytokine production and consumption activity in two separate regimes of concentrations of cy- respectively. Cytokines are produced by discrete cells, tokines [89]. Such dual feedback regulation of signaling and the production rate κprod(~r, t) concentrates on the was also shown to extend the range of cytokines that trig- cell surface. The consumption rate can be estimated by ger STAT5 phosphorylation while avoiding saturation, taking into account that cytokines bind tightly to cy- and ultimately control cell survival. The modeling of tokine receptors on the cell surface and get consumed such complex regulation of the JAK-STAT pathway in- by endocytosis of the engaged receptors. The limiting volves adding biochemical steps that can be integrated step for this cytokine consumption is then the binding to numerically. The challenge in this context is to acquire a receptors, so that large number of system-specific biochemical parameters. This task has become more amenable as the field pro- κconsum = konNRnconsumc(~r, t) = kcc(~r, t), (26) gresses and quantitative approaches have been delivering more and more estimates: protein abundances, biophys- where nconsum is the density of consuming cells. This ical parameters, enzymatic rates [78, 90–92]. expression is valid at low concentrations, when cytokine To conclude, in this section we have reviewed the ba- receptors are not saturated and the kinetics of binding sic equations governing the regulation of the cytokine- remain linear with the concentration of cytokines. It also activated JAK-STAT pathway in the immune system. assumes that consuming cells are uniformly distributed, While JAK-STAT signaling can be solved analytically in in a mean-field manner, rather than modelling the precise the adiabatic limit, the long-term dynamics of cytokine location of their surfaces. In more general cases, κprod accumulation and consumption are complicated by the and κconsum may have additional, nonlinear dependencies multitude of feedback regulations triggered in leukocytes. on c(~r, t), for instance because of spatial heterogeneity Such rich dynamical complexity will need to be tackled or receptor saturation. Little can be done analytically in quantitatively and systematically to deliver a more com- these cases and one must resort to numerical solutions prehensive understanding of cytokine communication in [93]. In the linear regime, Eq. 25 becomes: the immune system [75]. ∂c(~r, t) = D ∇2c(~r, t) − ξ−2c(~r, t) , (27) ∂t p B. Communication across space and time, and with a characteristic length ξ = D/kc over which cy- cytokine niches tokines diffuse before being consumed. In the following section, we focus on the simple homogenous and linear case to derive insights about the quantitative aspects of Cytokines allow immune cells to communicate and to cytokine communication. modulate their response collectively. In section V A, we will discuss how lymphocyte differentiation can be de- cided by toggle switches encoded in gene regulatory net- 2. Screening by cytokine-consuming cells works. Most of the positive feedbacks in these toggle switches are in fact associated with a response to cy- Diffusion and consumption of cytokines regulates how tokines, and are thus intrinsically collective. Tackling the the cytokine gradient spreads i.e. the lengthscale for nonlinearities and spatio-temporal aspects of cytokine cell communication. It was demonstrated that the short communication at a more quantitative level is critical timescale to reach stationary concentration profiles for to expanding our understanding how antigen recognition soluble proteins are explained by the first arrival time of leads to such a collective response. We now focus on the cytokine ligands, rather than the characteristic dif- the dynamics of cytokine secretion and capture in space fusion timescale for an individual molecule [94, 95]. This 19 result implies that the characteristic timescale to reach consuming cells “screen” the diffusion of cytokines from steady state at a distance r from the secreted cell scales the secreting cells and end up determining the extent linearly with r rather than with r2. The analytical ex- of cell-to-cell communication in a dense tissue. In other pression for this local relaxation time τ(r) is [94]: words, the simple diffusion-consumption of cytokines can account for the heterogeneous accessibility of cytokines 1 r τ(r) = 1 + . (28) in dense tissues and the formation of cytokine “niches,” 2kc ξ defined as the portion of space in where cytokines can be sensed. Two relaxation processes are at play in this system: the ξ is controlled by the density of cytokine-consuming 2 diffusion timescale (τd(r) = r /2D) and the reaction cells, and the number N of receptors on the surface of −1 R timescale (kc ). The relaxation timescale is dominated consuming cells. In close-packed tissues the density of by the reaction time at short distances (r ξ) where consuming cells is typically 1 cell every 10µm, imply- cytokine consumption dominates. However, at large dis- −3 −3 ing nconsum ∼ 10 µm ; for IL-2 D is approximately tances (r ξ), both diffusion and consumption are im- 100µm2/s (even in close-packed tissues, there is suffi- portant and the relaxation timescale is the geometric cient free space in the extracellular medium for cytokines −1 1/2 mean of the two, τ(r) ∼ (τd(r)kc ) ∝ r. Numeri- to diffuse freely as in solution), and each cell consumes cal simulations in the context of immune responses [96] typically 10 cytokine molecule per second [96], implying validated this theoretical argument and estimated that 11 −1 −1 konNR = 10 M s , so that ξ ∼ 25µm = 2.5 cell di- it takes only a few minutes for the cytokine gradient to ameters. In more physiological conditions, where cells are reach its steady state. more dilute, occupying only 10% of the space and cells The timescale at which immunological regulation by −2 −3 express lower level of receptors, nconsum ∼ 10 µm , cytokine exchange operates is typically in the range of this length scale rises to ξ ∼ 250µm ∼ 25 cell diame- hours for the activation of gene regulatory elements, to ters: there is negligible screening and all the cells within days for the decision to proliferate or to die. The typical a lymph node or a spleen have access to the cytokine. timescale at which cells move is also very slow. In the Note that this 10-fold increase in the screening length first phase of an adaptive immune responses, antigen- ξ implies an increase by 1000-fold in the volume of the responding T-cells are essentially stuck on their antigen cytokine niche and a 1000-fold increase in the number presenting cells. One can thus assume steady state for the of cells that potentially respond to the cytokine. Such distribution profile of the cytokines, c(~r, t) = c(~r), and tuneability can be critical to decide which cells (helper solve for dc/dt = 0 in Eq. 27. Using spherical symmetry T cells or regulatory T cells) win the tug of war for IL-2 c(~r, t) = c(r) for the Laplace operator, the steady state and ultimately whether the immune response expands or profile for the cytokine distribution is gets extinguished [77, 79, 80].

Rcell c(r) = c(R ) e(Rcell−r)/ξ, (29) cell r 3. Probability of autocrine capture and the boundary value c(Rcell) can be determined by estimating the flux of molecules secreted and consumed Cells that secreted cytokines often express the as- at the cell surface: sociated receptors, and can in principle communicate I with themselves. To quantify this effect, we must es- ~ 2 dc(r) κprod − κconsum = j.d~r = −4πDR timate the non-zero probability Pauto that cells cap- dr Rcell (30) ture their own cytokines rather than let them dif- R fuse and interact with neighbouring cells. This phe- = 4πDc(R )R 1 + cell . cell cell ξ nomenon, called autocrine signaling, is particularly significant when cytokine-secreting cells are surrounded by Typically, cells secrete cytokines at a constant rate be- cytokine-consuming cells and compete for their consump- tween kprod =10 to 1000 molecules per second (e.g. IL-2 tion. and IFN-γ cytokines) and consumption by the secreting At steady state, there is a balance between cytokine cell is given by kconsum = konNRc(Rcell). Solving for production kprod, cytokine consumption by the secreting c(Rcell) yields: cell kconsum, and diffusion away from the cell followed by consumption by other cells, as given by Eq. 30. The c(Rcell) = kprod/(4πDRcell(1 + Rcell/ξ) + konNR) (31) probability of absorption by the secreting cell is given by:

The characteristic scale −1 kconsum 4πDRcell(1 + Rcell/ξ) p p Pauto = = 1 + . ξ = D/kc = D/(konNRnconsum) (32) kprod konNR (33) is analogous to the screening length of electrostatic in- We can define the characteristic number of receptors at ∗ teractions and is inversely proportional to the square which this probability reached 50%, NR = 4πDRcell(1 + −∆Gassoc root of the density of consuming cells. By this analogy, Rcell/ξ)/kon ≈ Rcell/σR, where σR = Rreceptore 20 is the effective cross-section of the receptor-cytokine in- functional role in modulating immune responses. In the teraction (Eq. 2 and 6), and where we assumed ξ Rcell. first moments of activation, there are very few cytokine- Leukocytes are typically Rcell = 5 to 10 µm in radius, consuming cells, and the field of cytokines extends to ∗ 4 and σ ∼ 0.5nm, hence NR ≈ 10 . If cells have much fewer the entirety of the lymphoid organ (ξ > 100µm), reach- 4 than 10 receptors (a common situation), Pauto 1 and ing blood vessels and making communications global. most of the secreted cytokines diffuse away without be- Within 24 hours, many cells migrate closer to cytokine- ing captured in an autocrine manner. Alternatively, if secreting cells. As they do so, they increase their cy- 4 cells have more than 10 receptors, Pauto ∼ 1 and secret- tokine consuming capabilities by upregulating their re- ing cells capture a large fraction of their own cytokines. ceptors, thereby “screening” the diffusion of cytokines The functional relevance of autocrine capture remains de- (ξ ∼ 10µm) and limiting the extent of cell communica- bated in the field: why would a cell need to respond to tion to nearest neighbours, within a tight niche around the cytokine it secretes. There are also molecular mecha- secreting cells. nisms limiting its impact. For instance, T cells secreting IL-2 upon antigen recognition express low levels of IL-2 receptors as long as the antigen response lasts to limit V. CELL FATE autocrine capture [84]). In the tug of war between cell types for cytokine consumption, competition between autocrine and paracrine signaling is crucial, and thus the There is a large variety of types and subtypes of im- amount of autocrine capture is significant [80]. Discrim- mune cells, making the field of immunology sometimes inating between autocrine and paracrine signaling is of- difficult for the non-specialist. These fates are acquired ten difficult and it has been suggested that dilution ex- either at the very beginning during haematopoiesis, the periments in vitro can be used to distinguish the two process by which all types of blood cells are generated modes [97]. by differentiation from stem cells, or following an immune response or challenge, during which cells further specialize to better fight infections. We first describe the 4. Size of cytokine niches physical mechanisms of gene regulation that lead to stable distinct cell fates. We then review modeling strate- Eqs. 29-31 can also be used to estimate the functional gies for understanding and inferring the hematopoiesis size of a cytokine niche established by a single secret- differentiation tree. Finally, we discuss the problem of ing cell. In the limit of no autocrine capture and long differentiation during an immune response. dispersal ξ Rcell, we have: k c(r) = prod e−r/ξ (34) A. Gene regulation and cell differentiation 4πDr

The radius of the niche rniche can be defined as the Many cell decision making processes in the immune r at which c(r) falls below the characteristic concen- system are carried out at the level of gene regulation, trations KD needed to trigger cytokine signaling, and whereby expression of key transcription factors decide is the solution to the implicit equation: rniche = cell fate and subsequent immune effector function. For ξ ln(kprod/4πrKD). The logarithmic dependence on the example, T-cells sense their inflammatory environment rate of cytokine secretion kprod is significant because that as defined by the combinations of cytokines in the extra- rate may vary greatly depending on the cytokine or de- cellular environment, and drive their signaling response pending on the state of differentiation of cells. For ex- towards expressing key transcription factors that in turn ample, CD8+ T cells can secrete up to 1000 IFN-γ per decide which cytokines get produced and secreted. second, CD4+ T cells secrete 10 IL-2 per second upon To consider a concrete example of gene regulation, we activation from a naive state, and 1000 IL-2 per second use the classical example of the differentiation of CD4+ as they mature [84] or when get activated from a memory helper cells, a particular type of T cells whose function is state [98]. to regulate the action of other immune cells through the In Figure IV B 4, we present the numerical solution for production of cytokines — the messenger molecules used rniche for different secretion rates for cytokines with dif- by immune cells to communicate with each other. In their ferent screening lengths. Note how variable the volume of na¨ıve state, T cells are essentially a blank slate which the cytokine niche can be, based on realistic parameters can later differentiate into more specialized cell states to of immune responses. best match the pathogenic threat. Upon antigen stimu- To conclude, in this section we discussed how sim- lation, helper T cells can orchestrate three types of im- ple equations for the diffusion and consumption of cy- mune responses. They can elicit a ‘Th1’ response, further tokines in dense immunological tissues can quantitatively unleashing T- and NK cells’ cytotoxic response against account for the tunability of cell-to-cell communication infected cells. This is particularly relevant when eradi- in the immune system. It is particularly striking that cating an intracellular infection such as a viral infection. such active modulation of cell communication plays a Alternatively, helper T cells can elicit a ‘Th2’ response to 21

103 104 133 kprod=1 - screen =10 screen =10µm

kprod=1 - screen =30 screen =20µm

kprod=1 - screen =100 screen =40µm =80µm kprod=10 - screen =10 screen screen =160µm kprod=10 - screen =30 Radius of cytokine niche (µm) screen =320µm kprod=10 - screen =100

kprod=100 - screen =10

102 kprod=100 - screen =30 103 62 kprod=100 - screen =100

kprod=1000 - screen =10

kprod=1000 - screen =30

D kprod=1000 - screen =100

K Signaling threshold / ] e

n 1 2 i 10 10 28 k o t y C [

100 101 13 Volume of cytokine niche (pl)

10 1 100 6 100 101 102 101 102 103 Distance from cytokine secreter (µm) Rate of cytokine secretion (molecules per second)

FIG. 6. Size of cytokine niches for different parameters of secretion/consumption. (Left) Cytokine profile as a function of the distance from the surface of the secreting cell (normalized by KD – the critical cytokine concentration to induce STAT phosphorylation), for different rates of secretion kprod (in molecules per second), and different screening length ξ (in µm). (Right) Size of the cytokine niche as a function of the rate of cytokine secretion. For these graphs, KD = 3pMol and Rcell = 5µm in equation (34). engage with B cells, drive affinity maturation and class concentration. These nonlinearities and their functional switching in antibody production towards secretion of an- consequences for cell fate in the immune system can be tibodies that can annihilate extracellular (e.g. bacterial) studied using nonlinear stability analysis. Many tran- infections. Finally, T-cells can elicit a ‘Th17’ response scription factors auto-amplify their production in a direct that protects mucosal barriers from infection. These loop or, more generically, in a more convoluted cytokine- three characteristic types of T-helper cell differentiation mediated manner. For example, the transcription fac- respond to and amplify distinct cytokine environments. tor Tbet drives the secretion of a cytokine called inter- Note that similar decisions are made by other immune feron γ (IFN-γ) that signals through pSTAT1 to produce cells: for example macrophages differentiate into ‘M1’ more Tbet in the context of Th1 T-cell differentiation. In (pro-inflammatory) and ‘M2’ (anti-inflammatory) types, section XII B 1 of the Methods, we recall the basic for- which mirror the decisions and inflammatory outputs as- malism of nonlinear analysis, which explains the generic sociated with the Th1 and Th2 helper T-cell types. occurence of bimodal distributions of expression of transcription factors, cytokines and surface markers that im- Immunologists are interested in dissecting the inter- munologists encounter in their single-cell measurements play and feedback between inflammatory environments (cytometry or single-cell transcriptomics). We discuss and immune cell differentiation [99]. In that context, classical models of gene regulation and feedback regula- quantitative immunology brings the tools of nonlinear tion that account for the molecular programs enforcing dynamics to explain how sharp and cross-inhibitory deci- sharp cell differentiation in the immune system. sions can be made by activated T-cells during differentiation. Many immunological systems involve nonlinearities with self-reinforcing feedback loops, whereby the protein of interest acts as a positive feedback that drives further 1. Th1/Th2 differentiation expression. Additionally, most gene regulation involve mutltimerization of transcription factors onto one gene Early experimental evidence demonstrated that CD4+ locus, as in the case with phosphorylated STAT, a tran- helper T-cells commit to two distinct and incompatible scription factor induced by cytokine signaling. Multimer- states of differentiation, Th1 and Th2, upon antigen ac- ization implies a nonlinearity because the association rate tivation and response to inflammatory cues. A model of an n-mer scales with the nth power of the monomer’s describing this system is based on two inhibitory loops 22

0 P eq (see Sec. XII B 1) whereby transcription factor P1 inhibits where ωi = ωi + j ωj→iPj, Pi is the characteristic the production of P2 and vice versa (Figure 7A & B). concentration of Pi in differentiated cells at steady state, 0 n2 γi are degradation rates, ωi is the constitutive level of dR1 K2 = v1 − γR R1 expression of regulation of Xi, ωj→i is the (possibly neg- dt Kn2 + P n2 1 2 2 ative) influence of Xj on Xi, and σi is a parameter modu- dP1 lating the steepness of the regulation. Note that proteins = k1R1 − γP1 P1 dt can regulate their own production, ωi→i 6= 0. n1 dR2 K1 There are three main species involved in Th17/Treg = v2 − γR R2 n1 n1 2 differentation: P1=FoxP3, P2 =RORγt (the transcrip- dt K1 + P1 tion factors controlling Treg and Th17 differentiation, dP2 = k2R2 − γP2 P2, respectively), and P3 =TGFβ (a cytokine that regu- dt late T cell differentiation), which acts as an externally where R1 and R2 are number of mRNA leading to the fixed stimulus. Their interactions are represented in Fig- + production of P1 and P2, γ decay rates, ki and vi are ure 8A. For instance RORγt is activated by TGFβ production rates, Ki are binding dissociation constants, (ω3→2 > 0), as well as by itself (ω2→2 > 0), and but and ni are Hill coefficients. At steady state, is repressed by RORγt (ω1→2 < 0).

n2 Graphically, one can represent the nullclines (dXi/dt = K2 P1 ∝ 0) for both transcription factors as shown in Fig- Kn2 + P n2 2 2 (35) ure 8. There are three stable fixed points where the Kn1 + − P ∝ 1 , nullclines meet: RORγt FoxP3 for the Th17 cells, 2 n1 n1 + − + + K1 + P1 FoxP3 RORγt for the Treg cells and RORγt FoxP3 for a mixed state whose existence was later confirmed to which some background expression may be added to experimentally [103]. This formalism is interesting be- account for leaky gene regulation. cause it handles coupling and feedback phenomenologi- The nonlinearities, which are controlled by the Hill co- cally, through the effective regulation parameters ω , efficients n > 1, originate from the multimeric control i→j i rather than through detailed mechanisms of how these of transcription by the transcription factors. A graphi- regulations are implemented. cal inspection of the nullclines (Eq. 35) for this dynamical model (Figure 7.C) reveals that there exist three fixed points for this gene regulatory system, two stable high low high low 3. Experimental test of bistability in cell differentiation ones, (P1 ,P2 ) and (P2 ,P1 ), and one unstable high high fixed point (P1 ,P2 ). This simple dynamical system reveals how cross-inhibition of transcription factors The elegant models presented in the two previous sub- yields a classical toggle-switch that yields two incompat- sections (Th1/Th2 and Th17/Treg differentiation) ac- ible states. Note that such a model predicts hysteresis in count qualitatively for the observation that many acti- T cell differentiation (Figure 7D) as has been observed vation conditions lead T-cells to commit to clear, sepa- experimentally [100–102]. rate and stable cell fates. The crux of these models is the existence of toggle switches based on cross-inhibition of transcription factors and, optionally, self-reinforcing 2. Other differentiation switches positive feedback loops for transcription factors. Similar models have been proposed in the context of hematopoietic differentiation [104, 105], and they provide a quanti- A similar formalism has been applied to account for the tative framework to account for the bimodal distribution differentiation of CD4+ T-cells in Th17 and regulatory of transcription factors in immune cells. How do these T-cells (Treg). Th17 cells are pro-inflammatory and crit- models compare with experiments? ical to eradicate infections in mucosal tissues; regulatory T cells are anti-inflammatory and limit overzealous im- A theoretical inspection of the Th1/Th2 model of T- mune responses, such as auto-immune disorders caused cell differentiation by Antebi et al. [102] highlighted how by a strong response against self antigens, or runaway the bistable solution of the model presented in Figure 7 immune responses such as septic shock. These two states does not generically produce a bistable solution. A sim- of differentiation for CD4+ T cells can be accessed upon ple scan of parameters within the physiological range for exposure to the cytokine TGFβ. However, that cytokine induction of Tbet and GATA-3, the candidate P1 and has been shown to be both pro- or anti-inflammatory P2 in Eq. 35, demonstrated that the ratios of rates for depending on the context. Tyson & coworkers [103] pro- their self-reinforcement and inhibition most commonly posed a model to account for such bimodality in TGF-β predicted a single stable fixed point, and only rarely action. In that model, the concentration of each protein more than two. In addition, the induction of transcrip- species P generically follows: tion factors is known to be inherently stochastic, be- i cause of intrinsic noise in gene induction [106–109] or dX 1 extrinsic noise in the levels of expression of signalling i = γ Xeq − X (36) dt i i 1 + e−x i molecules in T cells [110]. Overall, the “simplest” model 23

A B Th0 (naïve T cell) v1 v2 Antigen R1 R2 Th2 helper cell helper Th2 Tbet TF1 Gata-3 TF2

P1 P2

Th1 helper cell IFN-g IL-4

C D g Stable Th2 - Stable Th1 =IFN 1

Unstable P 4 (Th2) - IL

IFN-g (Th1) v1

FIG. 7. Modeling Th1/Th2 differentiation. A. Cartoon model of the cross-regulation of Tbet and GATA-3 transcription factors (TF) whose expression determines T cell fate in terms of Th1/Th2 differentiation. B. Diagram of the interactions of TF deciding cell fates. C. Nullcline for the regulation of TF (P1 and P2): three fixed points can emerge in the dynamics of the system, two stable, one unstable. D. Dynamics of expression of P1 when the tonic rate of expression v1 (driven by antigen response) is increased: note the hysteresis in the levels of P1 whereby a bimodal distribution can be anticipated for intermediate values of v1.

B t A g TGFb Stable Treg

ROR Stable Th17 Unstable Th17 cell Treg cells Stable Mixed state RORgt FOXP3

FoxP3

FIG. 8. Modeling the trimodal distribution of T-cell fate upon exposure to TGFβ. A. Cartoon of the genetic network associated with the regulation of RORγ t and FoxP3 upon TGFβ. B. Nullclines for the dynamics of regulation of RORγt and FoxP3. Each intersubsection of the nullclines corresponds to a fixed point; graphical considerations can be used to deduce the stability of each fixed point. of a toggle switch in T-cell differentiation, though elegant in its inception, is not consistent with observa- 24

common lymphoid tions. This case is typical in quantitative immunology, progenitor (CLP) T-cell where initial insight is often derived from genetic ma- hematopoietic multipotent B-cell stem cell (HSC) progenitors (MPP) nipulations with severe consequences as in the case of Natural Killer (NK) cell the Th1/Th2 system where knocking out Tbet leads to megakaryocyte a reduction/maintenance of induction of Th1/Th2 cy- short term HSC erythrocyte (ST-HSC) tokine secretion, respectively [111]. These knockout ex- mast cell monocytes periments must be balanced and reinterpreted in non- common myeloid (macrophages, dendritic cells) perturbative settings, e.g. by monitoring the sponta- progenitor (CMP) granulocytes neous induction of Tbet in differentiating T-cells. Similar (neutrophils, basophils and eosinophils) reassessments are happening in many models of leuko- FIG. 9. Hematopoiesis. A cartoon representation of the cyte differentiation, such as in the M1/M2 model of outline of the hematopoiesis process that leads to the forma- macrophage differentiation [112], or PU-1-controlled dif- tion of immune cells. The cells that are discussed in some ferentiation in the myeloid compartment [113]). Future detail in this review are marked in red. work will require better parametrization an innovative modeling approach that can embrace the combinatorial and dynamic complexity of cytokine communications and lymphoids that include cells of the adaptive immune sys- gene regulation in the immune system. tem: B and T-cells as well as natural killer (NK) cells. Such “murkiness” in immunology must be embraced as The decision about becoming a myeloid or a lymphocyte it opens up functional possibilities. For example, Peine is widely assumed not be an autonomous decision but is et al. [114] tracked the formation of Tbet+GATA-3+ believed to be influenced by external signal (see below mixed phenotypes, i.e. with characteristics of both Th1 for a detailed discussion). and Th2, so in apparent contradiction with bistability. General quantitative questions about differentiation Their presence was demonstrated to limit the deleterious apply to the differentiation of cells in the immune system. impact of all-out, non-mixed Th1 or Th2 inflammation. For example, given that in mice there are of the order of Thus, a better understanding of noise and stability in 104 HSC, what kind of dynamics results in ∼ 107 very immune cell differentiation will be key to understanding short lived granolocytes (of the order of a day) while at virtuous and pathologic inflammation. the same time producing ∼ 1011 lymphocytes that can live even for years? How long does it take to produce these cells? What kind of differentiation process can pro- B. Hematopoiesis duce this kind of diversity, and how is it regulated to produce the right numbers of cells? A lot of information 1. Timescales about these differentiation processes has be gained from two types of experiments. The first involves exposing cells at a given upstream stage with died or radioactive Where do immune cells come from? A human being markers (i.e. bromium, deuterium) that get taken up has ∼ 5·1013 cells in the body and more than half of them — and then diluted — upon cell division. Analysis of are made by hematopoietic stem cells (HSC) found in the the decay curves is informative about cellular lifetimes. bone marrow. There are an estimated order of magni- The second type of experiment — an adoptive transfer tude ∼ 1016 cell divisions per human, which gives ∼ 106 experiment — is experimentally harder, since it involves cell divisions per second [115, 116], and most of them transplanting new marked cells into an animal (typically are linked to the HSC cells. HSC number ∼ 104 in mice mouse) and then tracking them. To study cells during (there are no reliable numbers for humans), many of them the early stages of hematopoiesis the mouse must first be already made in the fetus. Through a series of differenti- cleared of its natural cells, and recent results suggest that ation and phenotypic commitment events (HSC→ Short the dynamics of differentiation after transplantation may Term HSC (ST-HSC) → Multipotent progenitor (MPP), be very different from regular dynamics [117]. Lympho- these cells give rise to different kinds of cells found in the cyte divisions can be studied without killing the host’s blood, including red blood cells and all immune cells (see own immune system. The principle of these experiments Fig. 9). The first branching decision, whose precise tim- is simple — after some time δt the marked cell will appear ing is currently being questioned, is about becoming a in a given compartment. myeloid progenitor, that will give rise to red blood cells, At the population level, given that cells at stage i of mast cells, thrombocytes (so not immune cells) but also differentiation can proliferate with rate λ , differentiate macrophages, granulocytes (neutrophils, basophils and i with rate α or die with rate δ , the dynamics of a given eosinophils) and dendritic cells – so cells of the immune i i cell type n in the differentiation process follows [117, system that eat up cells and proteins non-specifically. R 118] They function as cells of the innate immune system, simply eliminating bacteria and other pathogens, but some dn R = α n − (α + δ − λ )n , (37) of them (e.g. dendritic cells) also play an important role dt u u R R R R in adaptive immunity as antigen presenting cells (APC). The remaining branch of the differentiation tree leads to where nu describes the upstream (pre-differentiation) cell 25 type. Cells can also migrate, which introduces a spa- these inverse trends of proliferation and self-renewal pro- tial component to the equations. In general, all these tect the organism from malignancy by making sure the rates are functions of the concentrations of the differ- right amounts of cells are produced. Myeloid lymphomas ent populations ni, resulting in non-linear equations. — a condition where the organisms produces too many αR + δR − λR = kR define a collective timescale of decay myeloid cells — have been linked to disturbances in this of the nR population. This highlights the identifiabil- balance and the condition is only visible in aged individ- ity problem that without specially planned out experi- uals when the overproduced cells have accumulated. ments guided by theory, it may be hard to tease apart the timescales of biological interest. Most HSC become myeloid cells, with only one in 100 cells becoming a lymphoid cell. Lymphoid cells are very Commonly in this kind of analysis it is assumed that long lived and the thymus can function for long time the cellular differentiation of the adult individual is in without input from new HSC. In general, due to the slow steady state. We will return to this assumption when we timescales of the whole process, the system has a lot of learn the timescales for the process. Within this assump- inertia. If all HSC are killed in a young adult mouse, tion the steady state ratio of cells in the compartment there are no visible phenotypic effects for ∼ 5 months, of interest, compared to the upstream compartment is unless the hemopoietic pathway itself is stressed [122]. nR/nu = αu/(αR −βR), with βR = λR −δR. This kind of approach has been used to learn proliferation rates for T- There are a priori no biological reasons for downstream cells [119]. Stochastic versions of such models have also cells in the hematopoietic pathway not to influence up- been considered in detail [119–121], showing that theo- stream cells, however this has not yet been directly ob- ries of age-structured cell populations [120] are equiva- served. Probably certain feedback mechanisms are at lent to branching processes [119] (see section XII C 6). play. For hemapoietic stem cells, a HSC in the bone marrow divides once every 50 days (λR = 1/110days) and a HSC becomes a granulocytes after ∼ 1 year (and then lives for a day). This is known from so-called fate-mapping experiments, consisting of tracking labeled cells, which is easier for granulocytes since they are known not to pro- C. Inferring the hematopoiesis differentiation tree liferate. It is also known that granulocytes have no other cell source, unlike other cell types. In this case the la- bel from the HSC appears in a granulocyte after a year. Analyzing experiments from mice with progenitor cells Since a lifetime of a mouse is ∼ 2 years, a 1 year timescale carrying unique barcodes with a maximum likelihood to produce a new cell may put the steady state assump- approach (see section XII E 1) coupled to a stochastic tion into question. The fetal dynamics is very different, model, Perié et al [123], uncovered a more complex the steady state assumption cannot be used, and a new picture of myeloid and lymphoid differentiation. They granulocyte is produced in a few weeks [117]. considered the lymphoid primed multipotent progeni- Technically, measuring cell counts directly is not reli- tors (LMPPs) differentiation process during which each able because of sampling bias. Instead, one should mea- LMPP is called an MDB, since it has the potential to be- i total sure cell ratios, fi = n /n , as was shown in the pre- come a myeloid (M), dendritic (D) or B cell. They built a vious example. In steady state, the fraction of cells of stochastic branching process model of cell differentiation a given type evolves according to dfi/dt = ki(fi−1 − fi), (see section XII C 6), where in each step of the decision with ki = αi − βi. By looking at subsequent compart- tree a cell can loose one or two of its potentialities: an ments it is possible to disentangle differentiation and pro- MDB can become an MD, MB, DB or directly an M, D, liferation. This gives an estimate of how self-renewing B. Similarly an MD can become an M or a D etc. They the compartment is (βi), and how much comes from up- contrasted this with a traditional differentiation model, stream differentiation (αi). For a perfect stem cell we where the MB phenotype is forbidden, and an MDB can- expect ki → 0. Real stem cells (e.g. HSC) are essentially not loose two potentials at once. Using mice with pro- self-renewing with τ (in mice) on the order of the life- genitor cells carrying unique barcodes, they build lineage time of the mouse. Other differentiation compartments trees, traced each offspring cell to its parent and counted −1 how many offspring of each type (N for i ∈ {M,D,B}) are clearly transitional, with ki ∼ 1 day. HSC divide i once every 110 days but they also differentiate once every were produced by each MDB cell. The generating func- P m d b 110 days, so they seem to have a lifetime of (they replace tion G(zM , zD, zB) = m,d,b zM zDzBP (m, d, b) for the themselves every) 110 days (this remains to be verified probability of a cell to give rise to a given combination of with more direct means). Short Term HSC (ST-HSC, see offspring cells P (NM ,ND,NB) can be calculated from a Fig. 9) die at a rate of 1 per month and Multipotent pro- convolution of simple branching process generating func- genitor (MPP) of 1 per day. This produces a strong self- tions [123]): renewal gradient from HSC to MPP. At the same time the proliferation rates of these cells go up from HSC to MPP with a similar gradient. Some have speculated that G(zM , zD, zB) = H(pMDB,GMDB(zM , zD, zB)), (38) 26 where H(p, z) is given by the generating function, in Lymphocytes, both T and B-cells, start growing and Eq. 130 in section XII C 6, and dividing upon stimulation during an infection. When X they divide they also acquire new specialized functions, GMDB(zM , zD, zB) = pMDB→izi make many decisions at the individual cell level, e.g. i∈{M,D,B} whether to divide further, whether to switch class for im- X (39) munoglobulins (Ig) molecules, whether to become mem- + p H(p ,G (z , z )), MDB→ij ij ij j j ory cells for both T-cells and B-cells, or to become plas- i,j∈{M,D,B},i6=j mablasts for B-cells. As their number increases, they pMDB, pMB, pMD, pBD are the probabilities of not dif- eventually make another decision to stop dividing and ferentiating from (staying in) the MDB,MB,MD,BD start dying. At the population level we observe a de- states and pMDB→ij describes the transition probabili- crease of the population size, after the large expansion ties from the MDB state to the ij ∈ {MD,MB,DB} due to proliferation, and a return to pre-infection cell states. The remaining functions are defined recursively counts. Since each cell can in principle adopt a differ- as in Eq. 39, e.g. for ij = MD: GMD(zM , zD) = ent decision path, at the population level we see a large pMD→M zM + pMD→DzD etc. combinatoric diversity of cell states: from cells that have The marginals of this distribution give the probabil- divided a small number of types and become memory ity that a given barcode appears in each of the 7 pos- cells, to cells that have divided many types to leave no sible cell types. Following the equal loss of potential offspring. model (ELP) [124, 125], the rate of loss of the cells’ abil- This diversity is even more staggering for B-cells, ity to make different cell types was assumed to be con- since they undergo a phenomena called class (or isotype) stant and independent of current cell type, which gives switching recombination (CSR), where the variable re- three rates αM , αB, αD, that parametrize the transition gion (discussed in Sec. VI) of the antibody does not probabilities between the cell types pi→j (for all allowed change but the constant region of the receptor is mod- combinations of i, j ∈ MDB,MD,MB,DB,M,D,B) ified. This process does not change the affinity of the and reduces the number of parameters. The parame- antibody for the antigen, but it means the antibody can- ters are determinied by carrying out a maximum like- not interact with signaling molecules (see sec. III B). B- P lihood fit ~pmax = argmax~p ( i niπi(~p)) , where ni cells do this by expressing a specific gene located in the is the number of observed barcodes in state i and heavy chain locus, organized in a specific order. During πi(~p) is the model probability (from Eq. 38) that CSR, segments of genes in this locus are removed, and an ancestral barcoded cell gives rise to offsprings in the remaining DNA is recombined to encode a different states i, where i ∈ MDB,MD,MB,DB,M,D,B cal- isotype. Since the double stranded breaks occur at con- culated from the generating function: πMDB(~p) = served nucleotide motifs, the identity of the expressed P P m,d,b≥1 P (m, d, b), πMD(~p) = m,d≥1 P (m, d, b = 0), isotype-encoding genes are conserved between cells and P πM (~p) = m≥1 P (m, d = 0, b = 0) etc. individuals. Since the non-expressed isotype-encoding This approach showed that the classical model of se- genes are deleted from the locus, in general the order of quential loss forbidding the MB state is not consistent cycling through the classes is conserved, although inter- with data. In fact, the MB state is very unlikely but chromosomal translocation from the other allele can add is essential to explain the observed barcode distribution. deleted loci [127]. Simulation of the dynamics of these models showed that How are these decisions made by cells? One view is the differentiation process between LMPPs and the com- that cells integrate signals from the environment, which mitted cell types requires ∼ 20 rounds of differentiation, act as cues to trigger decisions. While this is certainly which translates into about 2 weeks. Recent barcod- true [128, 129], and experimentalists can induce certain ing experiments combined with similar inference meth- cell fates in vitro using cytokine cocktails (see sec. IV), ods that looked at earlier stages of differentiation from an alternative (yet not contradictory) idea was proposed HSC cells [126] showed that most HSC cells give rise to in the “cyton” model [130]. The basic idea behind this many cell fates. model is that each cell in the population makes a stochas- Similar techniques of branching processes coupled to tic decision, choosing from one of the accessible cell fates. likelihood inference methods were initially used by Yates To do this it uses an intrinsic pre-programmed timer, et al [121], to analyse fluorescent dye experiments of dif- telling it when it should divide and die. However since ferentiating T-cells and estimate cell division and death the timer in each cell picks a time from a distribution of rates. times, each cell will go through a different decision scenario. Putting together the different decisions each cell can make this leads to a heterogeneity of cell fates in one D. Cell fate during the immune response population. Yet, for each trait (division times, becoming a memory cell), the population level distribution is 1. Choice and timing of cell fate under stimulation reproducible. The model can be and has been adapted to different cells [130–133] but let us present it on the simple example 27 of three possible options: to divide, to die or do nothing signaling molecules (cytokines), for which they need spe- even if stimulated. Given an experimentally character- cialized receptors. A cell that commits to participate in ized distribution of division time φi(t) and death times a given communication channel, expresses a surface re- ψi(t) for each division round i, and assuming that a frac- ceptor and the combination of surface receptors gives the tion Fi of cells will divide when stimulated in division spying experimenter an idea of how this cell is bound to round i, the number of cells dividing or dying per unit behave when triggered – this is what we call a cell type. time is a given round i = 1, ..., m of divisions is: This phenomenological approach has been very successful Z t Z t−t0 ! in the history of immunology and provides a way to clas- div 0 div 0 00 00 0 sify different cells and give us some idea about their func- ni (t) = 2pFi dt ni−1(t ) 1− dt ψi(t ) φi(t−t ), 0 0 tion and properties. In practice, FACS (Fluorescence- Z t Z t−t0 ! activated cell sorting) sorting experiments segregate cells die 0 div 0 00 00 0 ni (t) = 2 dt ni−1(t ) 1−Fi dt φi(t )) ψi(t−t ), according into ones that have a high concentration and a 0 0 low concentration of a given surface marker on their sur- where the terms in parantheses account for cells that face. By doing this in many dimensions, one can zero in would have divided (died) at time t but had previously on a very specific cell type. Technically, this leaves the died (divided). The total number of cells in each division problem of deciding where the boundary between high round is then calculated by summing gains and losses in and low is – in practice cells are heterogenous and the each round: experiment produces a distribution of marker concentra- Z t tions, which, if we are lucky, is bimodal. This problem 0 div die N0(t) = N − dt n0 (t) + n0 (t) , (40) of finding the separatrices between cell types is called 0 gating, and is an art implemented in analysis softwares. Z t 0 div div die Advances in machine learning will alleviate the need for Ni(t) = dt 2ni−1(t) − ni (t) − ni (t) , expert-guided gated and make the definition of cell types 0 more automatic and easy-to-validate. Being aware and for i = 1, ...m. These integrals can be performed analyt- using the heterogeneity of the population data is often ically for exponential distributions but since the experi- useful. Looking at the population distribution has made mental data are better described by log-normal distribu- people realize that often there is continuum of pheno- tions [134, 135], the integrals are performed numerically. types and a binary approach is not valid. This has been The cyton model and its generalizations have been amplified by recent high throughput cell sorting experi- shown to be able to fit the number of cells in experiments by mass cytometry –so-called CYTOF [136], which ments with and without stimulation [130]. Of course, one make it clear that cell types are more continuous and dy- could note that the cyton model takes the experimen- namic than traditionally assumed. With that in mind, we tally observed distribution of division and death times can now try to learn something about how cells acquire and simply calculates the number of surviving cells. Yet and switch identity. the power of the model lies in the paradigm shift that assumes there are rules and, while the immune system functions stochastically and relies on heterogeneity, we can nevertheless predict and understand its behaviour. 3. Inferring cell fate timelines during the immune response The idea of a programmed stochastic system, as opposed to a black box integration of cues, is extremely powerful. The problem of infering the timeline of cell differentia- Using data from transgenic mice infected with the in- tion during T-cell response can also be treated using the fluenza virus, Marchingo et al [131] showed that T-cell cyton theory [130–133]. Here we present another more receptors and costimulatory signals impose an intrinsic data-driven approach to the problem described in Buch- division fate on each cell, and cells then count through holz, Flossdorf et al [137]. We will concentrate on the generations, as defined by the cyton model, before re- example of CD8 T-cells. A clone, here defined as all turning to the pre-stimulation state. This initial heri- the TCR that respond to the same antigen, starts with table priming can be later modified by dose dependent ∼ 10−100 long-lived naive cells (lifetime of about 80 days cytokine signling, which is also integrated by the cells in mice). During an infection this clone grows to ∼ 107 and influences their the response. As a result different short-lived effector cells. After infection clearance, there combinations of costimulatory signal and cytokines can remains 103 memory cells with an intermediate lifetime generate signals of similar magnitude. These experiments (about 15 days in mice). These memory cells can then re- also show that the cyton model is not in contradiction spond more robustly in a subsequent infection. Memory with signaling-based decision making. and effector cells also proliferate within their classes to maintain their pool. This is done through small antigenic signals they constantly receive for memory, and strong 2. An aside on cell types antigenic signals for effector cells. The above textbook description provides a simplified Cell types in immunology are defined using surface picture, but is unable to put measurable numbers behind markers. Cells communicate by binding and unbinding each of the described populations. More importantly, 28 this picture was painted by looking by eye at bulk popu- model selection is then finalized by comparing the AIC lation data summing over many cells. Can inference and (see section XII E 1) of the best fit for each topology. The data discrimination approaches help us infer the differen- necessity to use correlations in the inference procedure, tiation pathways? Population-level adoptive transfer ex- and not just population averages, was demonstrated on periments can be used to infer the parameters of known synthetic data generated with known parameters and models, but it is much harder to infer the topology of the topologies. For instance, both a linear (A → B → C) network, especially from single snapshots. Flossdorf et and tree (C ← A → B) topology can give rise to the al [137, 138] analyzed single cell adoptive transfer (single exact same evolution of the mean number of cells as a cell fate mapping) experiments to discriminate between function of time, for well chosen parameters. However different differentiation tree topologies and find the un- covariances between these numbers give clearly different derlying model parameters. They considered a model signatures, allowing us to distinguish the different topolo- with four cell types: naive (N), effector (TEF), effector gies. These data-based, model-selection approaches are memory precursor (TEMp) and central effector memory similar to those developed for stem cell hemapoiesis [123], precursor (TCMp) and wrote down the most general net- described in section V C. work diagram, in which every non-naive cell type could As a result, from the 302 possible models, two strongly differentiate into any other cell type, all of which also stand out as potential candidates to describe the cell dif- could proliferate and further differentiate. Naive cells ferentiation of CD8 T-cells responding to a listeria ex- could become any of the other cells, but no other cell pressing chicken ovalbumin (to which these T-cells were could become a naive cell. This led to 302 possible mod- specific) in mice. These two models are: els, defined by their network topologies, as well as their parameters. The precise definitions of the particular cell types in the experimental data was done using surface naive → TCMp → TEMp → TEF, (42) markers (see discussion above) can be found in the origi- 10 % naive → TCMp → TEMp → TEF & naive −−−→ TEMp, nal paper. The immunological meaning of these cell types is under debate – but we are mainly interested here in the power of the theoretical method. The considered models where simple stochastic dynam- where each cell type except for the naive one proliferates. ics for each cell type that considered differentiation (~α) The models have well defined parameters that can easily and proliferation (β~) reactions. As mentioned above, a be identified. This bacterial response is an acute infection lot of heterogeneity was observed within naive cells and and generates exponential cell growth, and the inferred in vitro experiments show heterogeneity in the reaction proliferation rates form a gradient from TEF → naive rates, which had implications on the model. Differenti- cells with naive cells proliferating the least. Both of these ation rates where chosen from underlying distributions: models correctly predict the time dependent dynamics of an exponential and gamma distribution of differentiation the mean concentrations of the different cell types. Even rates were considered. The data resolution did not al- if learned on single-cell progeny data at only one time low to discriminate between the two, with an exponen- point, the first linear diversification model is able to pre- tial giving a sufficiently good fit. The proliferation rates dict the phenotypic composition of the expanding popu- were taken to be fixed, since longitudinal correlations lation at earlier time points. These kinds of approaches of proliferation events were previously shown to decay shed light on the possible differentiation dynamics and rapidly [139]. The considered models ignored the het- allow us to rule out possible scenarios. To give just one erogeneity in the phenotype of naive cells, but since the example, for this system the cell differentiation is largely heterogeneity in the number of naive cells would render (∼ 90%) symmetric: the two offspring of a cell are usu- model selection impossible, the experiment was tuned to ally the same, although not the same as the mother, as start with one naive cell (Nnaive = 1). opposed to asymmetric divisions where the two daughter The model consisted of a Master equation for the evo- cells have different cell types which happen about ∼ 10% lution of the joint probability of the different cell types: of times. Also, it turns out that the differentiation and proliferation rates must depend on time to explain the ~ ∂tP (N~ ) = F P (N~ ), ~α, β, nnaive, ~r , (41) data. These time-dependent rates are the same in a response to both bacterial (listeria) and viral challenges, where is F a linear operator describing the prolifera- suggesting universal response dynamics. tion and differentiation dynamics of the different cells Using similar methods to those proposed by Flossdorf types, N~ = (NTEMp,NTCMp,NTEF) are the numbers et al [137, 138], Miles et al [140] showed that the model is of cells of each type, and ~r is the death rate vector of consistent with the data of Buchholz [137] et al, but ap- all cell types. The analytically calculated first and sec- plied to the data of Kinjyo et al [141] suggests that mem- ond moments of these equations are fit to the variances ory precursors are produced before effector cells. This (CVi(t)) and covariances (Σij(t) and mean concentra- shows that this type of inference is robust but the bio- tions (hNii(t)) for each cell type i from cell fate map- logical interpretation of the results depends both on the ping data using a mean squared error minimisation. The experimental conditions and model assumptions. 29

4. Quorum sensing cells are also known to cluster in cultures and in-vivo. A two step model for the integration of these two op- Recent experiments have revisited the question of naive posing signals was proposed to explain the nonlinear re- T-cells differentiation into different memory cell types sponse of dendritic cells when integrating pro- and anti- upon antigen stimulation, by considering collective ef- inflammatory information [97]: an initial intermediate fects through cell-cell communication. Polonsky et al [86] molecule integrates both signals in an indiscriminate way tracked live cells tagged with antibody markers over time until a threshold value, providing a bottleneck. This way in microwells with different initial cell numbers. By con- either the pro- or anti-inflammatory signal can stimu- tinuously tracking individual cell differentiation states late the system. But the output of the this bottleneck is and proliferation, they are able to overcome the lim- later down-regulated by only the anti-inflammatory sig- itations of bulk experiments where local cell densities nal. This combination of collective decision making with are hard to control. Live-cell imaging showed that the a modulated bottleneck safeguard was proposed as a way decision to differentiate into progenitor central memory to control possibly excessive immune responses. (pTCM) is collective: in equal medium conditions, cells Collective decision making, also called “quorum sens- surrounded by more cells had a larger differentiation rate. ing”, was previously proposed theoretically as a way for That rate depended solely on the instantaneous num- immune cells to help solve the self non-self discrimination ber of cells in the well, as shown by data collapse of the problem [32]. According to that theory, which we discuss differentiation rate onto a single curve as a function of in more detail in Sec. VI C, communication allows cells the number of cells, for various conditions (different me- to make a census-based decision by integrating the sig- dia, different initial number of cells in the well). The nal read out by many clones in order to correct mistake curve shows that cells are more likely to differentiate into made by individual cells. This is a cellular implementation of an error-correcting code, similarly to kinetic proof pTCM if there are more than Nc ∼ 30 cells in the cluster. The process can be described by a stochastic differenti- reading discussed in section III A 1. The context of that ation model, in which cells divide and die, and differen- proposal is different from that of [86], where the T-cell tiate with a rate R. However, to explain the data R(N) population is monoclonal, while the idea of error cor- must be made to depend on the number of cells N. Such rection relies on taking a census of different T cell clones a stochastic model also captures the observed well-to- that make independent decisions. This difference in clon- well variability. The collective decision could be linked ality does not change the general similarity in the nature to the IL-2 and IL-6 cytokine communication: blocking of the collective decision. The experimental results are IL-2 reduced the maximum of the universal differentia- likely to hold for different T-cell clones with with similar affinities for the stimulating antigen. The T-cell response tion rate curve, while blocking IL-6 increased Nc without significantly altering the plateau value. Interestingly, IL- is multiclonal (see section VI G) and any quorum-sensing 2 and IL-6 receptors where found to cluster on cell sur- mechanism is likely to involve multiple distinct clones. faces, with receptor patches directed towards neighbour- Understanding the details of how the memory and ef- ing cells. This experiment shows that the percentage of fector cells pools are controlled in a multiclonal setting cells that differentiate into memory cells and effector cells remains an open question. depends on the instantaneous local T-cell density, independently of the additional post-stimulation influence of T-cell receptor signalling strength, or the effect of antigen VI. REPERTOIRES presenting cells. Polonsky et al [86] give a threshold in terms of abso- A. Size of immune repertoires lute number of cells in a micro-well needed for collective decision making, Nc ∼ 30. However, that number must For the adaptive immune system to protect us against depend on the size of the well, if cells do communicate all the different pathogens we may encounter, including through cytokine diffusion. As we have discussed in sec- ones that may not exist when we are born, we need a tion IV B in the context of CD4+ T-cell differentiation large set of different immune receptors. into Tregs and T-helper cells, signal propagate to short There are of the order of 4 · 1011 T cells circulating in distances as they are taken up by neighboring cells [96]. the human body [142], and of the same order of B cells For pTCM differentiation, this is facilitated by the IL-2 [143], each expressing a single type of receptor to a first and IL-6 orientation of the receptors towards secreting approximation [144, 145]. However, T and B cells di- cells. But at large distances, diffusion starts playing a vide, meaning that many cells can carry the exact same role. Active clustering seems to be a mechanism that receptor, defining a “clonotype”. The repertoire size, or puts cells in the non-diffusive regime. number of clonotypes, is therefore smaller than the num- The scale of cytokine communication has also been ber of cells. Early work [146] based on a subset of the shown to depend on the type of signal: an anti- repertoire gave an estimate of ∼ 106 unique β chains in inflammatory IL-10 signal produced by dendritic cells one human individual. is long-ranged, while the pro-inflammatory TNFα sig- However, these estimates were indirect, and have been nal produced by the same cells is short ranged. These updated recently thanks to the advent of repertoire 30 high-throughput sequencing (RepSeq) techniques in 2009 Shape space Key [147–149], discussed in a number of recent reviews [150– 155]. This method, which we briefly discuss in section XII F, allows one to obtain the sequences of a non- exhaustive but fairly substantial portion of all receptors immune in a biological sample. As different technologies have receptor been emerging, the need for standarisation of RepSeq data reporting has appeared, and the Adaptive Immune Receptor Repertoire (AIRR) community [156] has been organizing meetings and producing web-based and journal publications to promote data reporting unification. RepSeq experiments on human blood samples, com- cross- bined with statistical extrapolation estimators, report reactivity unique TCRβ numbers ranging from 4 × 106 [148] (using area Fisher’s Poisson abundance model [157]) to the order of 108 [158] (using the Chao2 estimator [159]). For BCR self antigen heavy chains, estimates obtained with the Poisson abundance model yields 1-2 · 109 [160]. One should take all these empirical estimate with great caution, as they make FIG. 10. The minimal repertoire size is set by the unverified, and probably wrong, assumptions about the number of self-antigens through negative selection.A two-dimentional cartoon of the recognition “shape space”, an shape of the clonotype abundance distribution when do- abstract space which summarizes the main bio-chemical prop- ing the extrapolation from small blood samples to the erties of antigen-receptor binding into a unique space [33]. entire organism (more on that in Sec. VI D). Theoretical Antigen-receptor pairs that are close are likely to bind and estimates based on population dynamics models assum- trigger an immune response, while distant pairs do not in- ing that naive cells divide little or not at all logically teract. Each expressed receptor of the repertoire covers a give much higher estimates, closer to the number of cells ball of cross-reactivity where antigens are recognized by that itself, 1010 − 1011 [161, 162]. receptor. Negative selection ensures that self-antigens (red How large is large enough? De Boer and Perelson [17] crosses) are not included in any of the cross-reactivity balls proposed that the minimal repertoire size is ultimately of the repertoire. To cover shape space efficiently with that determined by self-tolerance. We revisit their argument constraint, the repertoire size (number of balls) must scale in the light of modern estimates, with simplified nota- with the number of self-antigens to avoid. tions, starting with the easier case of B-cells. B-cells interact directly with antigens, unlike T-cells that inter- with an optimal selected fraction f ∗ = exp(−1) ≈ 0.37. act with short peptide fragments presented on the multi- This estimate suggests that the number of self-epitopes histocompatibility complex (MHC). Call p 1 the prob- is the main determinant of the minimal repertoire size, ability that a randomly chosen receptor recognizes a ran- with which it scales linearly. Another prediction is that domly chosen epitope. During the self-tolerance selection the recognition probability should be inversely propor- process, the initially generated repertoire of size R is re- 0 tional to the number of self-epitopes. We can intuitively duced to R, R = fR , where f is fraction of receptors 0 understand this result by thinking about the repertoire that survive negative selection, i.e. that do not recognize as a covering problem: viewing the set of epitopes recog- any self-epitope, so that nized by a single receptor as a ball in an abstract pheno- f = (1 − p)n ≈ e−pn, (43) typic space, how many such balls does one need to cover the entire space of foreign epitopes, while avoiding a fi- where n ∼ 105-106 is the number of self-epitopes (in hu- nite number of self-epitopes? The simple argument given mans). The probability that an epitope escapes the im- above, as well as the schematic of Fig. 10, tells us that mune system is then given by the volume of each ball, p, should be inversely proportional to the number of self-epitopes, n, implying in turn R −pR pe = (1 − p) ≈ e . (44) that their number (R) should scale with n. Empirical estimates suggest p ∼ 10−5 and n ∼ 105 − This gives the repertoire size R = −(1/p) ln pe as well as 106 [17], consistent with the theoretical prediction p ∼ the pre-selection repertoire size 1/n. The prediction of f ≈ 37% of cells passing nega- R = −(1/p)epn ln p . (45) tive selection is consistent with recent experimental es- 0 e timates 25% − 45% [163]. The predicted B cell receptor Assuming that evolution has optimized the recognition diversity crucially depends on pe, which is hard to es- probability p so as to minimize R0, solving for ∂R0/∂p timate. De Boer and Perelson estimated that 99% of an optimal p∗ = 1/n, and an optimal repertoire size antigens, presenting each 10 epitopes, were recognized by at least one antibody of the immune system. This ∗ 10 −2 ∗ 4 5 R = R0f∗ = −n ln pe, (46) implies pe ∼ 10 , and R ∼ 5 · 10 − −5 · 10 , which 31 is much smaller than current experimental estimates of 167]. These numbers are consistent with the theory p ∼ BCR diversity (∼ 109). 1/n. The fraction of negatively selected cells, predicted These estimates become a bit more involved for to be f ∗ ≈ 37% by the theory, is estimated to fall in the TCR since epitopes are presented by the Multi- range 20-50% [168]. As for B cells, predicting the size Histocompatibility Complexes (MHCs), which translates of the repertoire is hard without a good estimate of pe, into an additional level of sampling. As already men- but current estimates of R = 108-1010 are probably much tioned in previous sections, there are 2 types of MHCs larger than predicted by (48) regardless of the estimate relevant to adaptive immunity: MHC type I complexes of pe. present peptides from inside the cell to CD8+ (also called In summary, these theoretical estimates still seem to killer) T-cells, whose goal is to kill infected or cancerous be relevant today for the relationship between recognition cells; MHC type II complexes present peptides from out- probability, p, and number of self-epitopes, n, both for T side the cell to CD4+ (helper) T-cells whose role is to and B cells. Recent estimates of TCR and BCR diversity, stimulate B-cells during affinity maturation process (see however, are much larger than predicted by these simple section below). theories. However appealing, a major limitation of these Each epitope, or peptide derived from the antigen of estimates and predictions is that Eq. 43 assumes that interest (self or foreign) must both be presented by one each receptor is tested against all possible self-epitopes, of the m = 6 MHC genes for each MHC type, and be while in practice this is impossible due to the limited recognized by one of the R TCR expressed in the body. duration of lymphocyte maturation. We will come back This implies that the probability of escape reads: to this point in Section VI C devoted to modeling thymic selection. R m −pR pe = (1−q+q(1−p) ) ≈ exp −mq 1 − e , (47) where q 1 is the fraction of peptides that can be presented by a given MHC molecule. Eq. 43 is still valid, B. Inference of the stochastic repertoire generation process with n being the total number of possible MHC-self- peptide complexes. The repertoire size is then given by pn R0 = −(e /p) ln[1 + (ln pe/mq)], and the optimal size is Antigen receptors are proteins, which must be encoded 4 obtained through ∂R0/∂p: as genes in the DNA. Humans have of the order of 10 protein-coding genes. Directly encoding the whole di- ln p versity of immune receptor genes (∼ 108 − 1010) in each R∗ = −n ln 1 + e (48) mq genome would make it impossible for the DNA to fit in the nucleus. The immune system has solved that prob- with p∗ = 1/n and f ∗ = exp(−1). Thus, the scaling and lem by stochastically creating receptors in each cell, in a selection probability are predicted to be the same as for process called V(D)J recombination that combines com- B cells. binatorics and randomness to generate diversity. B- and The probability of presenting a given peptide on MHC T- cell receptors are made of two chains, light and heavy class I complexes is relatively well known: deep learn- for BCR, and α and β for TCR. The genome encodes a ing algorithms have been very succesful in building algo- certain number of gene templates, called V, D and J for rithms that predict which peptide is likely to be pre- the heavy and β chains, and V and J for the light and sented. These methods are implemented in software α chains. Upon creation of a chain, DNA is edited and packages such as netMHC [40, 164], which account for one of each of these gene templates is chosen per recep- known biological features to guide the learning. The tor. The combinatorics of templates typically results in same prediction task has been implemented for MHC ∼ 103 different receptors. To obtain the (much larger) class II [165], but the prediction is much harder, partly observed diversity, nucleotides are randomly inserted in a because unlike MHC class I presented peptides, which non-templated way and deleted at the junctions between have a relatiely fixed length of 9-11 amino acids, MHC the V and D and D and J genes (or V and J genes for α class II presented peptides have varying length, from 13 and light chains). This process of generating “junctional to 25. More data than is currently available is needed diversity” in fact accounts for most of the diversity of the to successfully train the neural networks, and to improve repertoire [169, 170]. prediction. The models predict that about 1% of pep- Characterizing the above described process of receptor tides are presented by any given particular MHC allele, generation in quantitative detail has been made possi- i.e. q = 0.01. ble thanks to the development of RepSeq methods (see The number of MHC-self-peptide complexes is given Sec. XII F). This can be done using out-of-frame se- by n = qmnp, where np is the number of peptides in quences (with a frameshift due to the random number the human proteome. Each individual has 6 MHC alleles of additions and deletions at the junctions), which are for each class, i.e. m = 6. np is roughly the number of nonproductive and hence a raw product of recombina- proteins, ∼ 3 · 104, times their average length, ∼ 400, i.e. tion, as they are free of selection effects. These sequences 7 5 np ∼ 10 , and n ∼ 6 · 10 . The recognition probability, survive selection because they are in the same cell as an p, has been estimated to be in the range 10−4-10−6 [166, in-frame sequence. Since each cell has two chromosomes, 32 out-of-frame sequences come from cells where the first the deterministic annotation of sequences impossible and rearranged chromosome resulted in an out-of-frame se- necessitating a probabilistic inference approach. A per- quence and the second rearranged chromosome resulted haps counterintuitive consequence is that the model pa- in an in-frame sequence. Due to the random insertions rameters can be inferred with arbitrary accuracy despite and deletions of nucleotides, it is impossible to reliably our inability to annotate any sequence reliably. determine how a given receptor was formed (V, D, J as- The results of the inference show that the probability signments, as well as distinguishing inserted from tem- of generation of receptors is incredibly reproducible be- plated nucletoides) from its mere sequence. Instead one tween individuals of the same species. The distribution can consider a list of scenarios (which includes V, D and is dominated by insertions and deletions, whereas gene J gene choice plus a number of insertions and deletions choice contributes relatively little to the overall probabil- at each of the junctions) and sum over them weighted ity. Among the features of V(D)J recombation, the gene by their probabilities as determined self-consistently us- choice distribution varies the most between unrelated in- ing a probabilistic model. Concretely, in the case of the dividuals [169], even when corrected for single nucleotide simpler α or light chains, the probability of a given re- polymorphisms (SNP) in V, D and J alleles [181]. As combination scenario r is: far as can be determined, the generations of the α and β chains are largely independent, and the overall gener- Prearr(r) = P (V,J)P (delV |V )P (delJ|J)P (insVJ), ation probability of a receptor is well approximated by (49) the product of the generation of its two chains [145]. Al- where delV and delJ denote the number of deletions at though no similar analysis has been performed for BCR, the V and J ends, and insVJ is the list of inserted nu- it is likely that the formation of its two chains are also cleotides. A similar expression can be written for the β independent. or heavy chains by adding the D gene and the related Although these recombination models ignore selection deletions and insertions. The model is the most general effects, to be discussed in the section below, it is a good factorizable distribution that is consistent with the data predictor of the abundance of particular TCRβ in a hu- and the known biological constraints (such as the relative man population [176], and it has been used to detect positioning of the genes in the genome). The overall gen- signatures of immune responses as deviations from this eration probability of a given sequence s is obtained by baseline distribution [182, 183]. summing of the probabilities of all scenarios that could have given rise to this sequence: X C. Thymic selection and central tolerance Pgen(s) = Prearr(r). (50) r→s After the receptors are generated, they undergo an ini- The learning of the model parameters, encoded in the tial selection step, known as thymic selection in T-cells. probability distributions in the factorized form (49), is A similar process called central tolerance occurs in B- performed by maximizing the likelihood of the sequences, cells maturation [163], but let us focus on T-cell for con- using the Expectation-Maximization algorithm to deal creteness. For a more comprehensive survey of models with the sum over the hidden variable r (see Sec. XII E 1 of thymic selection, we refer readers to the review by A. for details about Maximum Likelihood and Expectation- Yates [168], and focus here on recent development involv- Maximization). ing quorum sensing and data-driven models of sequence This inference procedure has been applied to variety of specific selection. immune receptors and species, TCR β [169] and α [171] T-cells mature in the thymus, an organ that con- chains, BCR heavy [170] and light chains [172] in humans, tains only proteins that are native to the host organ- as well as TCRβ in mouse [173] and BCR heavy chains in ism, termed self-proteins. The newly generated receptors trout [174]. It is implemented in the IGoR software [175] are expressed on cell surfaces and their binding proper- which can be used to learn additional models from other ties are tested against the peptides from the self-proteins, species and locus combinations. IGoR can also be used to presented by the MHC complexes. If the receptor fails generate synthetic sequences from the Pgen distribution, to bind any self-peptide, even weakly, it will probably and to estimate Pgen(s) of an arbitrary nucleotide se- fail to bind any protein and the cell carrying these re- quence s. Another method, OLGA [176], was designed to ceptors is discarded – a process called positive selection estimate the probability of amino-acid sequences, using which removes ∼ 80% of immature cells. Conversely, if dynamic programming to deal with the enormous sums a receptor binds any one self-protein too strongly, it is involving the enumeration of all possible nucleotide vari- also discarded as a result of negative selection, since it ants. Other methods relying on hidden Markov models is likely to bind self-proteins later on and trigger auto- (see Sec. XII E 1), were also proposed to handle the high immune diseases. Negative selection was at the core of dimensionality of hidden variables [177–180]. the argument for the optimal recognition probability and Repertoires generated in silico using the learned model repertoire size presented in Sec. VI A, and it is estimated confirm that the inference is able to call the correct re- to remove 50% − 80% of cells. The detailed process of combination scenario in at best ∼ 25% of cases, making T cell selection, complete with the timing of the α and 33

double double negative naive the two distributions are well separated, giving the con- TCR positive selection dition Rpx2 1, with an optimal discrimination thresh- negative positive CD4 CD4 Progenitor cell ∗ cell selection old t = Rp(1 − x/2) in the limit x 1. This condition to is barely satisfied by x = 10−2, R = 109 and p = 10−5, periphery for which the optimal threshold for the number of TCR β, γ, δ α naive rearrangement involved is t∗ ∼ 104. This argument suggests another, rearrangement CD8 CD8 stricter lower bound on the diversity R of the T cell reper- spleen thymus - cortex periphery toire that is necessary to make such a collective decision. thymus - medulla It also implies a “quorum sensing” mechanism by which responding T cells have a way of estimating how many FIG. 11. Thymic selection. A cartoon representation of other cells are involved in order to commit to a response. the basic processes that take the cells from the bone marrow Recent work suggests that such quorum sensing does oc- to the thymic cortex and medulla and finally lead to the gen- cur locally, probably using cytokine signaling [86]. eration of functional naive CD4+ and CD8+ T-cells that are To model thymic selection in more quantitative detail, exported to the periphery. The duration of the processes is a common strategy has been to use additive models of not drawn to scale – it is meant solely as an indication of temporal order and overlap. binding free energy similar to (9) [29, 30, 35]. In these models, the only receptors that survive are those whose maximal affinity E∗ to any self peptide falls within a β chain recombination events, is summarized in Fig. 11. range (Ep,En) corresponding to the positive and nega- During positive selection, cells with TCR that recognize tive selection thresholds. This condition can be mapped peptides presented by the MHC class I commit to be- onto an extreme value statistics problem, allowing for a coming CD8+ cells, whose main function is to kill in- statistical mechanics treatment [31]. Under this frame- fected cells, while cells with MHC class II TCR specificity work, it was shown that the sequence composition of neg- become CD4+ cells, whose main function is to help B atively selected TCR was biased towards weakly binding cells in their affinity maturation, and to regulate the im- residues [29], and the theory was subsequently applied to mune response. There are about 4 times as many CD4+ explain clinical data on “elite controllers” of HIV express- cells as CD8+ cells. A small subclass of CD4+ T cells, ing a particular type of MHC class I molecule, HLA-B27 called regulatory T cells (Tregs), actually suppress the [187]. A similar theory was used to study the sensitivity immune response, and play an essential role to prevent of TCR that target tumor cells [30]. The parameters of auto-immunity [184]. Tregs are selected for higher affini- these models are not inferred empirically, but are instead ties to self-epitopes than regular T cells, which allows picked from popular but unrealistic interaction potentials them to selectively suppress immune responses to self- between amino acids such as the Miyazawa-Jernigan ma- antigens, although the picture seem to be quite complex trix [28]. Yet many conclusions of these studies are rela- (see [185]). tively insensitive to the details of the interaction matrices in the extreme value statistics regime. To be able to ensure self tolerance, each receptor Statistical inference methods based on immune reper- should in principle be tested against all possible self- toire sequencing can also be used to estimate the prob- epitope, which would take an impractically long time. In ability that a particular receptor passes selection based practice, experiments suggest that each T cell may en- on its sequence. We can define a selection factor corre- counter around 500 antigen presenting cells [186], while sponding to the ratio of probabilities to find a sequence theoretical arguments estimate the number of presented in a selected repertoire, P (s), relative to its probability self peptides around a few thousands [32] (these numbers sel in the unselected repertoire, as given by the recombina- are not inconsistent because each antigen-presenting cell tion model P (s): Q(s) = P (s)/P (s). In practice, may present many different peptides in single encounter gen sel gen it is impossible to evaluate P (s) directly, as the num- with a T cell). These numbers are much lower than the sel ber of possible sequences to be considered is too large, estimated diversity of presentable self peptide-MHC com- spanning more than 20 orders of magnitude in genera- plexes, 5 · 105 − 5 · 106 (see Sec. VI A). Calling x the tion probabilities. However, simplifying assumptions can fraction of presented self peptides during negative selec- be made on the form of Q(s). Specifically, Elhanati et tion, the probability that none of the selected TCR are al [188] considered: self-reactive is (1 − p)RNx, which quickly goes to 0 as R becomes even moderately large, even if p scales as 1/N. L A recently proposed solution to this problem is to as- Y Q(s) = q(L)q(V,J) qi|L(ai), (51) sume that a minimal number t of TCR must recognize i=1 a peptide to trigger an immune response [32]. By virtue of the law of large numbers, the number of T cells re- where (a1, a2, . . . , aL) is the amino-acid sequence of the sponding to a self-peptide is distributed as a Gaussian Complementarity Determining Region 3 (CDR3, which with mean and variance = Rp(1 − x), while the number forms a loop important for antigen recognition, and cov- of T cells responding to a foreign peptide is of mean and ers the most variable part of the receptor ranging from variance = Rp. Good discrimination is achieved when the end of the V to the beginning of the J segments). 34

Selection is assumed to act independently on the length it determines the amount of overlap one expects between of the CDR3 region q(L), the V gene, q(V ), and the repertoires of distinct individual, and underlies the con- J gene, q(J). The D gene is not taken into account cept of “public repertoire” shared by all individuals (see seperately, but selection on amino acids in the CDR3 Sec. VI E). All these aspects can be formalized within is considered explicitly. The parameters of the model the common langage of statistical mechanics through the are inferred by maximizing the likelihood using Expec- definition of a density of state, which we introduce below. tation Maximization (Sec. XII E 1). This approach has The most general family of diversity measures for a been applied to the initial selection process of β-chain se- distribution p(s) is given by Renyi entropies: quences [188], α-chain sequences [171], and BCR heavy " # chain [170] and light chain [172] sequences. For many 1 X H = ln p(s)β . (52) of these cases, the Q(s) selection factors did not dif- β 1 − β fer substantially from individual to individual. Instead, s they reflected an global selection for general biophysical This definition reduces to the Shannon entropy for β = 1, and biochemical features of amino acids (suggesting pos- P H1 = − s p(s) ln p(s) (see Sec. XII E 5), to the total itive selection for proper protein function), rather than number of receptors for β = 0, R = exp(H0), and to the individual-specific removals in the repertoire caused by probability of drawing the same receptor twice with re- negative selection. Similar observations were made for P 2 placement for β = 2, exp(−H2) = s p(s) , also equal to BCR by direct comparison of pre-mature and mature the inverse of the the Simpson index. More generally, the repertoires [189]. Modeling negative selection is much family Hβ recapitulates the entire clone size distribution, harder, as it requires to learn the “holes” that selection defined in its cumulative form as the number G(E) of re- pokes into the repertoire, whose post-selection landscape ceptor sequences with − ln p(s) < E, through a Laplace looks like a dense golf course. A simple way to model transform: negative selection is to remove a random fraction 1 − q of Z the sequences from the selected repertoire, boosting the 1 −βE Hβ = ln dG(E) e . (53) likelihood of surviving ones by a factor 1/q [190]. In prac- 1 − β tice, in simulations one can use a hashing function, which associates a quasi-random number to each sequence, and Note that G(E) is formally equivalent to a cumulative select sequences whose hashing number falls below a cho- density of states in statistical physics. E = − ln p is sen threshold, so that selection is reproducible, yet ag- sometimes called the “surprise” and is also formally sim- nostic to the features of the sequences. ilar to an “energy” by analogy with Boltzmann’s law, p ∼ exp(−E), making β analogous to an inverse tem- Note that models of the form (51) are not restricted perature. The mapping between G(E) and H is for- to thymic selection, and can be used to describe the β mally analogous to the equivalence of the canonical and sequence-wide selection pressure during any process, by micro-canonical ensembles, and is amenable to a statis- comparing the initial distribution before selection to the tical physics analysis [191]. The lower the temperature final distribution after selection, as was for instance done (the higher the β), the more the Rényi entropy concen- to characterize responsive receptors to yellow fever vac- trates on low-energy, high probability sequences. cination [183]. For instance, different T cell subsets (e.g. G(E) can also be interpreted as the rank (ordered from CD4+ and CD8+ cells), different BCR isotypes, in dif- most probable to least probable) of sequences with prob- ferent organs or environments, are probably character- ability p = e−E. The distribution of clone sizes is often ized by distinct selective pressure, each of which could presented in terms of a clone-size frequency rank distri- be modeled using similar Q(s) factors. bution, where clones are ranked ordered by their sizes, and their normalized frequency is plotted as a decreasing function of its rank. G(E) precisely encodes this rank- D. Diversity and the clone size distribution frequency relation. It is important to distinguish between the poten- The distributions of TCR and BCR span a very large tial diversity, corresponding to a theoretical distribution space, and are highly skewed. It is often useful to sum- Pgen(s) or Psel(s), and the realized diversity, correspond- marize these distributions by a single summary statistics ing to the actual distribution of a finite number of recep- that quantifies their diversity. Diversity measures in the tors in a particular sample, p(s) = n(s)/Ntot, where n(s) context of immune repertoires have been thoroughly dis- is the number of molecules or cells with receptor sequence P cussed in a recent review [162]. Diversity is important s, and Ntot = s p(s). for understanding how well our immune repertoire pre- Let us start with the potential diversity derived from pares us against a wide range of pathogenic threats. As the probability distribution Pgen(s) (see Sec. VI B). For we will see, diversity measures are deeply linked with the human TCR β, the total number of possible sequences, distribution of sizes of immune clones (number of cells eH0 , is infinite for all practical purposes (> 1039 [162]), as with the same receptor), which contains important infor- it is larger than the total number of TCRβ receptors hav- mation about the dynamics of immune cells in response ing ever been produced by a human being. For this rea- to environmental challenges (see Sec. VII). In addition, son, it makes more sense to compare Shannon entropies. 35

arise from noise is the Polymerase Chain Reaction, which 10 5 )

p ampliﬁes diﬀerences exponentially [196], were put to rest (

G thanks to the introduction of unique molecular barcodes , 10 4 associated to each initial mRNA molecule [154, 197, 198]. > p Specifically, the clone size distribution follows a power- law: 10 3 ˆ 1 G(E) ∼ eβE ∼ , (54) pβˆ 10 2 with β ≤ 1 but close to 1. The probability distribution of clone frequencies, 10 1 dG 1 ρ(p) = − ∼ ˆ , (55) number of clones with frequency 1+β 10 0 dp p 10 -6 10 -4 10 -2 10 0 clone frequency p is also a power law of exponent slightly below 2. Another way to look at the distribution is to consider the inverse FIG. 12. Distribution of clone sizes follow power laws. relationship, Cumulative distribution G(p) of clone frequencies p of unfrac- 1 tionned TCR β sequences sampled from the blood of 6 human p ∼ , (56) donors [171]. The dashed line represents slope −1. G1/βˆ which also follows a power law, called Zipf’s law in this particular context when its exponent βˆ−1 is close to 1. These can be computed using (53), where G(E) is evalu- Since G is the rank of the clone (from most frequent to ated from the probability density distribution of E among least frequent), this relation is called the rank-frequency randomly drawn sequences according to p(s), ρ(E) = relationship. −E R E 0 0 0 (dG/dE)e , so that G(E) = 0 dE ρ(E ) exp(−E ). Because of these long tails and limited sampling, very Entropy estimates for both nucleotide and amino acid few of the diversity measures can be estimated reliably. sequences are reported in [176]. Human nucletoide TCR One way to overcome these issues is to build a statisti- β diversity is H1 ∼ 44 bits (bits refer to ln(2) units) cal model that is then parametrised by the experimental for nucletoides and 30 bits for amino-acids, TCR α di- data. In Sec. VI A we already mentioned Fisher’s Pois- vesity is ∼ 30 bits for nucleotides and 25 bits for amino son abundance model and the Chao estimators, which acids. The BCR heavy chain entropy is 67 bits for nu- use this idea. More sophisticated methods have been cleotides and 53 for amino acids. These numbers are very proposed for immune repertoire, e.g. using Expectation large. If the distribution were uniform, that would cor- Maximization (see Sec. XII E 3) [199] or using rarefac- respond to 255 ≈ 3.6 · 1016 distinct amino acid TCRαβ tion curves [200], but all these methods are susceptible sequences. Because the distribution is non-uniform, the to huge errors when the clone size distribution follows pooled repertoire of all humans having ever lived do not a power law [201]. We expect the reported numbers to exhaust the potential diversity. As noted before, most grossly underestimate the true diversity of both BCR and this diversity is due to random insertions and deletions. TCR. To make progress, models of lymphocyte popula- Thymic selection and central tolerance effectively re- tion dynamics should be leveraged to reliably extrapolate duce these potential diversities in Psel(s), by around 9 the tails of clone-size distributions. We will discuss some bits for TCRβ [188], 4 bits for TCRα [171], and 12 bits of these models in Sec. VII. for BCR heavy chain [170], but the corresponding diversity number remain very high. This diversity loss does not mean that e.g. 2−13 ≈ 10−4 of TCR must be dis- E. Repertoire sharing carded; the probability of thymic selection Q(s) is in fact correlated with generation probability Pgen(s), meaning Receptor sharing is at the heart of the public vs private that diversity is reduced by selecting already likely se- debate in immunology. Public receptors are those shared quences. between individuals in a given cohort, while private are Let us now turn to the realized diversity in a sample ones that are seen only in individuals. If one studies a or an individual. This diversity is best described by the group of individuals with a specific condition (e.g. cy- whole clone-size distribution encoded by G(E). Based on tomegalovirus or CMV [204] or Ankylosing Spondylitis recent high-throughput sequencing experiments, the dis- [205]), receptors that are shared between these people tributions are often long-tailed, spanning several orders can be expected to be linked to a disease. This justi- of magnitude (see Fig. 12) [162, 192–194]. These tails fies the search and investigation of “public” sequences. seem to be mostly due to the memory fraction of the Both humans and mice do share a non-negligible numbers repertoire [195]. Initial worries that these long tails may of receptors in functional repertoires, regardless of their 36

A. B. m. The ensemble of sequences shared between m samples thus corresponds to sampling the generation probability at temperature 1/m, focusing on more and more likely sequences as m increases. distribution of antigens of This calculation shows that in such large ensembles of cells as the B or T cell repertoire, which contain upward of 108 unique receptors, we expect the most common receptors with a high probability to be independently gen- numbersequences of erated multiple times in diﬀerent people. The number

of receptors of of shared TCRβ sequences among sampled repertoires

number of individuals optimaldistribution from a cohort of 658 human donors [204] was well pre- sharing the sequence shape space dicted by the model in Eq. 57. By itself, the generation probability Pgen was enough to build a classifier that can FIG. 13. Overlap of immune repertoires between in- determine whether a given sequence will be shared by a dividuals. A. The amount of shared lymphocyte receptors minimal number of people, with a > 95% accuracy for (TCR β chain) between individuals is consistent with pure m > 10. This analysis also brings our attention to the chance [190] (data from [202]). B. To explain that observa- definition of public repertoires. A receptor that is shared tion, a simple model of optimal immune coverage of pathogens [203] predicts that even very similar immune environments ex- between m = 2 and m = 1000 people has a very dif- perienced by two individuals (top) can lead to very different ferent notion of publicness. For this reason it is better (and peaked) distributions of protections (bottom). to talk about degrees of publicness. In addition, sharing and publicness depends very sensitively on the collected sample size Ni. A reasonable choice for a single defini- lifestyles, environmental factors and family ties. How- tion of receptor publicness could that the sequence must ever, receptors are also expected to be shared by chance, be present on average once in each individual, implying because some sequences are likely to recombine indepen- Pgen(s) > 1/R. dently in different individuals [188, 206–208]. One can Interestingly, the theory can also identify receptors try to correct for this convergent recombination by using that are shared for other reasons than sheer chance. an outgroup — a group of individuals that are as simi- These receptors have a high propensity to be shared, but lar to the condition-specific group but do not have this a relatively low generation probability. The discrepancy condition — and identify sequences shared more in the between chance prediction and observed sharing was used studied group than in the control group, as was done for to discover shared T cell clones in identical twins [171], to CMV [204]. Alternatively, one can use the knowledge of find candidate TCR sequences associated with conditions immune repertoire diversity encoded in Pgen(s) to esti- such as CMV or diabetes without a control group [183], mate our null expectation. or to assess the public BCR response to vaccination in All sharing properties are recapitulated by the cumula- trout [174]. tive density of states G(E). Given n independent reper- Identifying common, even unusual receptors does not toires of sizes N1,...,Nn, the expected number of se- necessarily mean they are responding to the same anti- quences shared by exactly m repertoires, Mm, is given gen or pathogen, and even when it does it does not tell by the following generating function [190]: us what the target antigen is. As discussed in Sec. II D,

n immune response is a complex phenomenon that involves X g(x) = M xm (57) the binding different epitopes to immune receptors, fol- m lowed by signal propagation, cell commitment, and cell- m=0 cell communication through messenger molecules. Yet Z +∞ n Y h −E −E i = dG(E) e−Nie + 1 − e−Nie x tools based on the statistical expectations of an unbi- 0 i=1 ased repertoire allow us to identify deviations from this baseline, and propose candidate responding receptor se- For example, for n = 2, in the limit of small samples, quences to be tested in other experiments. −E N1,2e 1, the expected number of shared sequences is Z −2E F. Optimal receptor distribution M2 ≈N1N2 dG(E) e X (58) In Sec. VI A we discussed theoretical arguments for ≈N N p(s)2 = N N e−H2 , 1 2 1 2 an optimal repertoire size, but recent data suggests that s actual repertoire sizes are much bigger than these theo- reducing to simple biased birthday problem related to retical predictions. However, as stressed in Sec. VI D, the Simpson’s diversity and thus Rényi entropy of order 2. distribution of receptors is itself highly skewed, suggest- More generally, sharing between m small samples is gov- ing that the relative abundance of receptor types, rather erned by R dG(E) e−mE and to Rényi entropies of order than their absolute number, is an important factor of 37 repertoire design. Can we find an optimization principle they pave the way towards a better characterization of over the composition of the repertoire? Take a distribu- the specificity, reproducibility, and publicness of the im- tion Qa of antigens to be recognized, where a lives in a mune response at the repertoire level. A common obser- theoretical high-dimensional “shape space” encompass- vation is that the response is mostly private, although ing both antigens and receptors, and in which proximity responding clones are more shared than would expected reflects receptor-ligand affinity [33, 209], through a recog- by chance. Responding sequences also tend to be clus- nition or “cross-reactivity” matrix far between antigen a tered in sequence space [218]. This clustering gives a and receptor r which encodes the probability that a ran- criterion for identifying responding sequences in a sin- dom encounter between the two results in an immune gle repertoire snapshot, based on the density of similar response. For a given distribution of receptors, Pr. The sequences (differing by at most one amino acid) in the expected cost of an infection linked to a is a decreasing repertoire, relative to this density expected by the re- P 0 function c(·) of the probability that a chance encounter combination model, 0 0 Pgen(s ) [183]. P s ,|s−s |≤1 with a receptor is successful, r farPr, so that the expected cost for a random infection reads [210]: VII. LYMPHOCYTE POPULATION DYNAMICS ! X X C({P }, {Q }) = Q c f P . (59) r a a ar r T and B cells are organized in clones of cells that a r express the same immune receptor. These clones grow Minimizing that cost with respect to the receptor dis- and decay as individual cells divide or die either sponta- tribution {Pr} gives an interesting multi-peak structure neously or in response to external signals — e.g. an anti- (Fig. 13B). Because of the degeneracy of recognition al- gen recognition event triggering cell proliferation, or cell- lowed by cross-reactivity, similarly well protecting reper- cell communication through messenger molecules called toires in different individuals may exist, even if they sam- cytokines. The correct way to approach and to model ple essentially the same pathogenic environment (Figure these complex lymphocyte dynamics, as well as their im- 13B). As a result, we should not be surprised by great dif- plication for experimental observables such as clone size ferences at the phenotypic level, let alone at the sequence distributions or clone expansion, is still a largely open level, when looking at repertoires responding to specific question. There is a vast body of literature on model- threats. In other words, even if convergent selection is at ing lymphocyte dynamics for broad subpopulations with play, each immune repertoire may find a different molec- no clonal information, using models ordinary differential ular solution to it, which may explain the observation equations. Ref. [120] provides a useful entry point into that most repertoire sharing seem to occur by chance. that literature. Here we will focus on stochastic models population dynamics, where the identities and sizes of individual lymphocyte clones are tracked. This approach G. Repertoire response to an immune challenge is in line with recent developments in high-throughput repertoire sequencing (Sec. VI), which allows for such RepSeq technologies allow one to track changes in a fine-grain description of population dynamics, and for the repertoire in time and in response to environmen- which much work remains to be done, both experimental changes, infections, or vaccinations. In many animal tally and theoretically. While progress on this topic is models such as mice or fish, it is generally not possible still in its infancy, it promises to give us better insight to sample the repertoire in the same individuals. In- into the collective decision-making and dynamics of pop- stead, the repertoires of distinct but isogenic individu- ulations of immune cells, and to extract important in vivo als are sampled at the different times before and after parameters from data, at a moment when most studies an immune challenge. By construction, this strategy re- are based on cultured cells or mouse models. stricts the analysis to public features of the repertoire re- We also briefly review the application of differential sponse. To identify common sequences features of reper- equation models to HIV dynamics, in what constitutes toires that responded to a particular challenge, machine perhaps the most successful application of modeling to learning techniques based on low-level features [211] or translational immunology. support vector machines [212] have been used in mice to classify responding repertoires. Enrichment of particular V classes and sequences features have also been reported A. Neutral dynamics in the BCR repertoires of trout following immunization [174, 213]. Let us fist consider a simple stochastic model of reper- In humans, the BCR response to influenza vaccination toire evolution in the absence of any antigenic prolifera- [197, 214], BCR and TCR response to varicella-zoster tion. Such a model is relevant for the naive pool, which vaccination [215, 216], and the TCR response to yel- is often believed to be unaffected by external stimuli (allow fever vaccination (a model for an acute infection) though this is also debated, see below), but can also serve [217, 218] have been studied at the repertoire level us- as a “neutral” baseline against which to compare other ing RepSeq. These studies are mostly descriptive, but dynamics or longitudinal data. While some of the results 38 discussed in this section were presented in [194], here we be simply neutral as modeled by Eq. 62, but must be expand on them and add a few original results, notably affected by selection, such as expansions events following on the establishment of the steady state, and on estimates antigen recognition, as we will see in the next section. of the total number of clones. The dynamics of Eq. 62 can in fact be solved analyti- In this simple model, new clones of size C come out cally using generating functions and the method of char- of the thymus (for T cells) or bone marrow (for B cells) acteristics [222]. Assuming for simplicity that clones have P with rate θC , so that the total cell output θ = C CθC . initial size 1, θC = θδC,1, we obtain (see Sec. XII D 3): Then each cell can divide with rate ν, and die with rate C µ > ν. θ 1 ν C 1 − e−t(µ−ν) The total number of cells C (t) follows the simple NC (t) = . (64) tot ν C µ 1 − (ν/µ)e−t(µ−ν) differential equation: dC This relation is still a power law with an exponential cut- tot = θ − (µ − ν)C. (60) off. The power law exponent remains 1 at all times, but dt the cut-off gets larger and larger as the system reaches This simple equation is at the basis of many studies aim- steady state. The total number of clones, Ntot(t), then ing at quantifying lymphocyte population dynamics, us- evolves according to: ing experiments with isotope labeling [219, 220] or other markers of cell divisions [221]. When thymic output, di- θ µ − νe−t(µ−ν) N (t) = ln , (65) vision and death rates are constant, Eq. 60 is solved by tot ν µ − ν (assuming no cell at t = 0): P and the total number of cells, Ctot(t) = C CNC (t) fol- θ −t(µ−ν) lows Eq. 61. At steady-state, the total number of clones, Ctot(t) = 1 − e . (61) µ − ν Ntot = (θ/ν) ln[µ/(µ − ν)] depends on both the division and death rates, and not just their difference. In the limit At steady state, the total number of lymphocytes Ctot = of small division rate, ν µ, which is often assumed for θ/(µ−ν) reflects the balance between thymic output, cell naive cells, we get N = θ/µ ≈ C . Conversely, in division and death. It is believed that the division and tot tot the limit µ − ν µ, one has Ntot ∼ Ctot ln(1/) with death rates ajust themselves so as to keep Ctot constant, 6 = (µ − ν)/µ, meaning that the number of clones may a process called homeostasis. For T cells, θ ∼ 10 /day be arbitrarily smaller than the number of cells. Start- for mice, and θ ∼ 108/day for humans, although that 11 ing with initial clone sizes of k0 does not substantially number varies with age. Taking Ctot ∼ 10 , this gives −1 affect this picture, with Ntot ∼ (Hk0 /k0)Ctot (where an effective decay time of (µ − ν) ∼ 1, 000 days, which H = Pn 1/i) in the limit ν µ, and with unchanged is consistent with lifetime estimates of T cells in human n i=1 scaling Ntot ∼ ln(1/)Ctot in the limit µ − ν µ (see using deuterium water [220]. Sec. XII D 3). However, these numbers are not informative about the These models can be refined by accounting for con- repertoire structure and its clone size distribution. These vergent recombination, by which clones sizes can also be calculations are also unable to disentangle division and increased by thymic exports that have the exact same death, lumped in the single parameter µ − ν. The ex- sequence, which happens with probability P (s) for a pected number of clones of size C, N (t), is governed by gen C particular sequence s. With this correction, Zheng and the following equations [193]: collaborators [223] found fair agreement between the results of a neutral model with a source Eq. 62 and the bulk dNC =ν((C − 1)NC−1 − CNC ) distribution of clone frequencies observed in the naive dt (62) repertoire of mice. However, they also found many out- + µ((C + 1)N − CN ) + θ C+1 C C liers, i.e. large naive clones that cannot be explained by the neutral assumption. A major difficulty of such a At steady state, dNC /dt = 0, the solution for C > comparison is that, as we already mentioned in Sec. VI D, max{C : θC > 0} is a power-law of exponent 1 with an exponential cutoff: the clone frequency distribution of the full repertoire is heavy-tailed. Any small contamination of non-naive cells 1 ν C into the studied repertoire is likely to introduce spurious N ∼ . (63) C C µ outliers in the distribution, confounding the analysis. When birth and death are balanced, ν ∼ µ, the exponential cutoff disappears. Comparing to the power law B. A note about “neutral processes” Eq. 55 in the distribution of frequencies ρ(p) with the cor- respondance NC = Ntotρ(p = C/Ctot), this model would In this review we discuss four different types of pro- predict a exponent βˆ = 0, which is not supported by cesses that are referred to as neutral in different do- repertoire data on unsorted or memory cells. This sug- mains: (i) the neutral model of population genetics (see gests that the dynamics of the memory repertoire cannot section XII C 8), (ii) the neutral clone size distribution 39

Neutral model NC Comment Eq. the assumption that new types arise with a rate pro- Lymphocyte dynamics ∼ (µ/ν)C /C Source Eq. 63 portional to the population size is incompatible with the Population genetics ∼ 1/C Fixed pop. size Eq. 141 biology of T cells, as new clones originate from the thy- Yule process ∼ 1/C1+ρ Expanding pop. Eq. 207 mus, and not from already circulating cells. On the other hand, the Yule process may be appropriate for modeling TABLE I. Summary of different neutral models discussed in population-wide affinity maturation of B cells, as their the review, with their clone size distributions. ν < µ refer to receptors hypermutate upon expansion. the division and death rates respectively, and ρ = 1/(1 − α), where α the probability of mutation upon division in the Yule process. C. Population dynamics model with external signals model (see section VII A), (iii) the Yule-Simon process of To explain the power laws observed in the clone size speciation (see section XII D 5) and (iv) the neutral eco- distribution from sequenced repertoires, one needs to go logical model (see section XII D 1). Since these models beyond neutral models and introduce external signals af- are often confused with one another, sometimes simply fecting the division and death rates — or fitness — of cells for semantic reasons, otherwise because many of them as a function of their phenotypic state or of their immune give power law distributions for some observable, we receptor. Several models of clonal dynamics have been thought it is pedagogical to discuss how they are dif- proposed to describe the growth and decline of clones in ferent and summarize the exponents they give and what various populations (see section XII D 1 for a general dis- kind of distributions they predict. cussion of competition models). We will not describe all Many of these models predict a power law in the distri- of these models, but try and give the reader an idea of bution of clone or species frequencies, but details differ. their general features. The key ingredient is that cells di- Table VII B gives a summary of these distributions and vide in response to antigenic stimulation, which depends differences between the model assumptions. on the concentration or frequency of available antigens Neutral models of ecology come in different flavours that are susceptible to be recognized. Cells in a clone ex- [224], but the simplest one is essentially equivalent to pressing receptor r die with constant rate µ as in Eq. 62, Eq. 62, and the corresponding clone size distribution, but they divide with clone-specific rate called the species abundance distribution in that context, ! has long been known as Fisher’s logseries [157], althouth X X Fisher derived it through purely statistical means using νr ({Qa}, {Cr0 }) = ν Qafr,aA Cr0 fr0,a , (66) unjustified assumptions. a r0 The neutral model of population genetics is similar to where f is the cross-reactivity function between an the neutral model of Eq. 63, with a major difference: r,a antigen and a receptor, Qa the frequency of antigen a, the population size Ctot is fixed. Most often, novel types and A(x) is a decreasing function describing the avail- arise from mutations arising in existing types. In popula- ability of a given antigen as it is being bound by other tion genetics the “site-frequency spectrum” corresponds receptors than r; for example, A(x) = (1 + )/(1 + x)2 to the distribution of allele frequencies, and is thus the with setting the strength of competition. In this model, equivalent of the clone size distribution NC . In the receptors interact only indirectly through this competi- neutral model, the site-frequency spectrum is a straight tion factor. power law of exponent −1. The difference with the result As a historical note, models with direct interactions be- of Eq. 63, beyond the absence of an exponential cutoff, tween receptors, namely antibodies binding to each other, is that mutants that reach the (fixed) population size were once proposed as a way to generate interesting dy- C become the wildtype, and are thus removed from tot namics on so-called idiotypic networks [225]. While id- the dynamics of mutants, while in lymphocyte dynamics iotypic networks were often used to explain phenomena no single clone ever takes over the whole population. in the 1980’s, and became very popular with physicists The Yule process is a model of an ever expanding pop- because of their link to network theory and the physics of ulation with mutations. It assumes that individuals of disordered systems [209], it is not currently considered a all types divide with rate µ, with a mutation probability dominant paradigm in immunology (and already was not α that generates new types. The crucial difference with by the time Ref. [209] was written, as discussed therein), Eq. 63, beyond the absence of death, is that the rate of as its predictions can be explained by clonal selection novely increases linearly with the population size. Also, theory alone without invoking additional elements. the population expands exponentially, and the clone size In the linear noise approximation, the size of the clone distribution only reaches a quasi steady state, after renor- of cells expressing receptor r is described by a stochastic malizing the total population size, Ctot. The renormal- differential equation: ized size distribution asymptotically follows a power law of exponent −2 in the limit α 1. Although this ex- dCr = [ν ({Q }, {C 0 }) − µ] C + ξ (t), (67) ponent is tantalizing close to those observed in data, dt r a r r r 40

0 0 with hξi(t)ξi(t )i = (νr ({Qa}, {Cr0 }) + µ) Crδ(t−t ) with If we assume many small expansion events, ∆x 1, Itô’sconvention. This general form of the model can de- expanding the exponential at second order allows us to scribe a wide variety of situations. It can be used as a recover te power-law exponent of [193]: model for the naive T cell repertoire, where Qa is the µ − sp∆x distribution of self-antigens for which T cells compete α = , (72) for survival cues. Such models of competitive exclusion sp∆x2/2 [226] have been studied and used to explain biological where the numerator is the net decay rate of the clone size facts for both T- [227] and B cells [228] repertoires, such obtained from the difference between death and expan- as the fact that a larger diversity is beneficial in terms of sion, while the denominator corresponds to an effective overall repertoire fitness. Interestingly, the same dynam- diffusion coefficient stemming from the random arrival of ical equation emerges as a way to approach an optimal expansion events. repertoire [210], as defined in Sec. VI F. The result of Eq. 69 suggests that a wide range of Eq. 67 can also be used to model lymphocyte popula- models with clone-dependent, random expansion or selection dynamics in response to transient antigenic stim- tion events produce power laws with arbitray exponents, ulation. Antigens appear with some rate s, and de- in agrement with the data on memory or unfractioned cay as they are being cleared by the immune system as lymphocyte repertoires. To distinguish between differ- Q (t) = Q0e−λ(t−ta), where t is the time at which anti- a a a ent types of dynamics, more details of the dynamics of gen a appeared in the population. In this context, the individual infection events, as well as of the kinetics of antigen is not limiting, and is fully available, A(x) = 1. unstimulated clones should be studied in terms of their When λ is very large compared to µ and ν, each clone impact on the clone size distribution. experiences a series of short spikes of expansion of mag- The hypothesis that the expansion factor ∆x is con- nitude exp(νhQ0i/λ) with rate sp, where p is the proba- stant, or at least does not depend on the clone size, bility that f = 1 for a random choice of a and r (and ar is questionable. For instance, in secondary immune re- f = 0 with probability 1 − p, in an all-or-nothing ap- ar sponses, the expanded cells already have a memory phe- proximation). With the change of variable x = ln C, each notype, and these cells tend to expand less than naive expansion event causes an average jump ∆x = νhQ0i/λ. cells upon a primary infection. In addition, the amount In Ref. [193], expansion events were assumed to be of inflammatory signals that promote expansion should both small and frequent compared to other time scales, in general depend on the number of cells involved in the which led to an effective diffusion equation for x. Here, response. From a design perspective, it would seem ben- as an original result of this review, we present the so- eficial not to expand clones that are already big, as the lution to the general process. A more detailed solution organism is already well protected against the pathogens using Laplace transforms is presented in Sec. XII D 4). that these clones are specific to. The optimal expansion The density of clones with logarithmic size x, ρ(x, t) = x strategy upon each pathogen encounter can be calculated (dC/dx)N x (t) = e N x (t), follows the simple jump C=e e within the simplified framework described in Sec. VI F. process with constant negative drift: Using Eq. 59 with a logarithmic cost c(x) = − ln(x) and ∂ρ ∂ρ a uniquely specific Kernel far = δa,r, combined with = µ + sp [ρ(x − ∆x) − ρ(x)] + θ˜(x). (68) Bayesian prediction theory, the optimal dynamics can be ∂t ∂x shown to approximately follow [229]: where θ˜(x) = exθ(C = ex) is the density rate of new dCr X clones entering the population with initial logarithmic = χ δ(t − t ) − τ −1(C − χθ ), (73) dt i m r a size x. Assuming a power law Ansatz for the clone size i∈Er distribution, −1 where τm = 2τ/(χ Ctot − 1) (with τ the effective 1 timescale of the pathogen dynamics), Er is the set of in- NC ∝ , (69) C1+α fection events i occuring at time ti in response to which lymphocytes carrying receptor r expand, and χ is a scal- −αx translates into exponential decay in x, ρ(x) = ρ0e , ing parameter setting the total number of lympthocytes. giving the consistency equation for α: The main difference between these and previously considered dynamics is that the growth of Cr is not exponen- µα + sp [1 − exp(α∆x)] = 0. (70) tial, meaning that regulation or homeostasis mechanisms must exist to tune the magnitude of the expansion as The total numbers of cells and clones can be calculated a function of the size of the antigen-specific repertoire at steady state (see Eqs. 204,205 in Sec. XII D 4), and subset. read in the case of fixed introduction size θ = θδ : C C,C0 So far we have assumed that selection manifests itself at the level of clones: all cells in the same clone has the θ ln(C ) θ(C − 1) 0 0 same average growth rate. However, selection could be Ctot = ,Ntot = ∆x . (71) sp∆x − µ sp(e − 1) − µ cell dependent, which is the case if cells respond differ- ently to non-antigenic stimulation signals, such as cy- 41

tokines. Growing experimental evidence shows the het- germinal center erogenous response to cytokines, due to differential ex- antigen plasma internalization antigen binding selection B-cell pression of cytokine receptors, signalling molecules and antigen dendritic cell competition for their diffusion. Models with a cell-dependent fitness do T-cell binding light zone not give power law behaviour, but do produce long tails memory helper [193]. With the current experimental cut-offs, it may B-cell T-cell not be possible to discriminate between clone-specific x and cell-specific fitness models using repertoire data only. naive Further studies with longitudinal tracking of clone sizes B-cell may help settle these questions. activated naive B-cell dark zone

somatic hypermutations D. In host HIV dynamics.

Perelson and collaborators [120, 230] considered a FIG. 14. Affinity maturation in germinal centers. Naive modified SIS model of an HIV infection. This is one B-cells get recruited into germinal centers where they become of the most influential examples of how computational activated and acquire somatic hypermutations upon proliferation in the dark zone. This produces cells with different models influenced medicine. It was used to predict the BCRs. If the BCR manages to get expressed on the cell sur- effects of anti-viral drugs on HIV: reverse transcriptase face, it moves to the light zone to undergo selection for binding (RT) which blocks the ability of HIV to infect a cell, and to antigen presented on follicular dendritic cells (FDC). Cells protease inhibitors (PI) that result in the production of that bind strongly internalize the antigen and present them non-infectious viruses. Target cells T , that correspond to helper T-cells. They undergo a second selection step where mainly to CD4+ T-cells, become infected (I) at rate β, they compete to helper T-cell signals. The sucessful cells can are born with rate α and die with rate µ. Infected cells undergo another round of affinity maturation by returning die at rate ν, but they also help the virus (V ) reproduce into the dark zone to somatically hypermutate, or they can and produce p new virions that are cleared with rate c. leave the germinal center as a memory B-cell or an antibody secreting plasma cell. dT dI dV = α − µT − βIS, = βV T − νI, = pI − cV. dt dt dt To study the effect of an antiviral drug, the virus equa- VIII. AFFINITY MATURATION tions are modified to account for two viral species: non- infectious viruses (V ) and infectious viruses (V ): NI I Upon antigenic stimulation, B-cell receptors acquire dV dV somatic hypermutations (SHM) that can help them ex- I = (1 − )pI − cV , NI = pI − cV , dt PI I dt PI NI plore the binding landscape. Based on high-throughput repertoire sequencing data, combinations of clustering where β = 1 − RT and 0 < PI < 1 and 0 < RT < 1 are and tree building methods [233, 234] have been proposed efficacies of RT and PI drugs. The total viral populations to characterize lineages. Since affinity maturation is a is held constant VNI + VI = V = const. For PI = 1 and fascinating example of a Darwinian evolutionary process, RT = 1, assuming T = const, the total viral load will a lot of effort is going into extracting the details of the decay according to [120, 230]: evolutionary processes from data, along with more the- cV (t = 0) oretical efforts. We very briefly review tree building, V (t) = V (t = 0)e−ct + × (74) c − ν clustering and lineage reconstruction approaches that cV (t = 0) are available in different software in section XII C 11. {e−νt − e−ct} − νte−ct . A recent review more thoroughly sumarizes these at- c − ν tempts [235], arguing for methods departing from the tra- This decay was fit to data from an HIV-infected patient ditional assumptions of population genetics, and tailor- on anti-viral therapy by adjusting the c and ν param- made for the hypersomatic mutation scenario. BCR eters using least-squares regression, showing very good repertoire sequencing provides large amounts of data that agreement [231]. Explaining viral dynamics in the case allow software packages like Immcantation [234, 236– of combination therapy (tri-cocktails) requires introduc- 240], Partis [179] (see section XII C 11) SPURF [241] or ing long-lived infected cells that act as a secondary vi- IGoR [175, 233] to learn hypermutaiton models (as de- ral source [232]. Fitting these more complex, yet still scribed below), and understand the evolutionary process. extremely simple models to data, allowed Perelson and As mentioned in section XII C 11 the problem of lineage collaborators to estimate the half-lives of the different reconstruction is quite general to many areas in immunol- species in Eq. 74. This in turn made it possible to ef- ogy. Incorporating the observation that more abundant fectively administer a tri-therapy cocktail treatment in clones are likely to have more offspring allows for recon- intervals that made it very hard for the virus to escape. structing B-cell lineage ancestries from germinal center This extremely simple calculation saved lives. imaging experiments [242]. 42

Affinity maturation is a sped-up Darwinian evolution where: ~τ j is the germline (unmutated) ancestor of ~σj; process. Upon recognizing an antigen even with low affin- I~τ j (i, ~π, b) is an indicator function that is equal to 1 if ity [243], naive B-cells migrate to germinal centers (GC) ~τ j matches π over the position range (i − 2, . . . , i + 2), in lymph nodes and about 3 days after antigen injec- j ~π and τi → b is a synonymous mutation; finally sb = ~π P ~π ~π ~π tion the germinal center reaction starts [244], initially N / 0 N and s = 0, where N is the total num- b b 6=π0 b π0 b dividing without hypermutations (see section V D 1) for ber of synonymous mutations to b observed in 5-mer mo- about 7 days and reaching a population size of ∼ 1500 tifs ~π across all sequences in the training dataset. cells [243]. About a week after the infection starts, they The S5F model provide a profile of hypermutation hot first enter the dark zone region of the germinal center, and cold spots in the absence of selection, which is widely where the Activation-induced cytidine deaminase (AID) used for analyzing BCR data. In particular, it can used enzyme introduces hypermutations as the cells continue as a baseline for quantifying selection from synonymous to divide. mutations [246, 247]. Note that a 5-mer model was also learned from VH and JH genes sequences in rearrangements engineered not to be productive, in both the heavy A. Modeling the hypermutation process and light chains of mouse, without having to use the trick of using synonymous mutations [239]. Repertoire sequencing analysis of unselected muta- To avoid for over-fitting, one can further assume that tions [170, 175, 234] has shown that AID acts highly the impact of the motif on mutability is additive. This non-uniformly, with hypermutation hotspots and strong strategy was applied to non-productive heavy-chain se- context dependent motifs. The difficulty in learning quences in humans [175]. Mutability of (2m + 1)-mer these models stems from eliminating selection that acted motifs ~π is given by: on memory sequences. A 5-mer model, called S5F, Pm was learned from synonymous mutations (that are in µ exp( i=−m e(πi)) principle free of selection) from long heavy chain se- Pmut = Pm , (78) 1 + µ exp( e(πi)) quences [234]. Only half of possible 5-mers were ob- i=−m served from data with synonymous mutations, the re- where (π−m, ..., πm) is the 2m + 1-mer sequence con- maining ones were inferred by averaging over related text around the mutation i, and e(πi) are the in- observed pentamers. Alternatively, out-of-frame mem- ferred elements of the Position-Weight Matrix (PWM), ory sequences were used to learn heavy V and J seg- learned within the IGoR framework [175] (see Sec. VI B) ment di- and tri-nucleotide context dependent hypermu- using the Expectation-Maximization algorithm (see tation models from low throughput sequencing experi- Sec. XII E 1). ments [245]. The actual hypermutation process is yet more compli- The S5F model learns the probability Pmut(~π) of cated than captured by all these models. It has been mutating the central position of a 5-mer motif ~π = pointed out that because of the context dependence of (π−2, . . . , π2) by counting occurences of synonymous mu- mutations, the order in which they appear matters. Since tations in this motif centered at position i along the se- considering all possible histories is computationally not j quences ~σ of a large dataset. Because only synonymous feasible, Feng et al proposed to Gibbs sample the order- mutations must be taken into account, ad hoc heuris- ings to learn a hypermutation model that was validated tic normalization rules were used. In practice, mutation on simulated data [248]. More importantly, hypermuta- rates are given by: tions are not independent of each other, and tend to clus- m~π ter along the sequence [175]. Hypermutations result from Pmut(~π) = µP (75) a lesion of DNA followed by error-prone DNA repair over ~π0 m~π0 extended regions (∼ 20 nucleotide) of the sequence [249]. with µ an adjustable baseline mutation rate (not learned These events may induce several simultaneous mutations within S5F), and at close-by positions along the sequence. It also sug- j j gests that context can influence the mutation rate over 1 X C /B X m = ~π ~π Cj (76) fairly large distances, since the erroneous repair can occur ~π P j P j j ~π0 j,π0 C~π0 j ~π0 C~π0 /B~π0 ~π0 far from the original damage causing the lesion. Finally, context models cannot explain all of the variance of hy- is the expected number of mutations of each 5-mer in a permutation rates, suggesting that other factors, such as sequence. This number is obtained by normalizing for length dependence, may be at play. being synonymous in each sequence, and then averaging over sequences weighted by their number of synonymous j mutations. In that expression, C~π is the observed number B. Cycles of selection of synonymous mutations in sequence ~σj in motif ~π, and Bj is a background normalization: ~π After acquiring hypermutations B-cells pass to the j X j j X ~π light zone of the germinal center where they undergo C~π = I~τ j (i, ~π, σi ),B~π = sb I~τ j (i, ~π, b) (77) i i,b the selection step of Darwinian evolution. In this step 43 a B-cell receptor must bind the antigen presented on a duced and the mouse was challenged with either bacteria follicular dendritic cell (FDC) strongly enough for the B- or virus antigens, the GCs showed great heterogeneity re- cell to internalize the antigen and present them to helper gardless of the type of antigen - from nearly monoclonal T-cells. B-cells compete for binding of their pMHC to to ones with an effectively neutral distribution of lineage these helper T-cells that give them an essential survival diversity. The GCs expressing one lineage can be traced signal. Therefore each B-cell passes a two-fold selection back to one single ancestral BCR (with possible SHM) step: selection for recognizing an antigen, and selection that expands in a clonal burst over a short period of time, for competitive binding to a helper T-cell. Binding to leading to a loss of diversity. Interestingly, the affinity of T-cells is specific and depends on the peptide presented BCRs coming from GCs with varying levels of lineage on the MHC. B-cells that have a higher affinity for the diversity is similar. The oligoclonality of antibody reper- antigen, have a larger internalization rate and a larger toires is also confirmed by RepSeq experiments that show probability to succesfuly solicit T-cell help. Helper T-cell that antibodies responding to a specific antigen found at stimulation is essential for survival. One unverified hy- physiologically relevant concentrations can have between pothesis is that helper T-cells that have previously been 3 − 147 distinct CDR3s [259]. trained in the thymus against self-proteins, now discriminate against hypermuations that lead to self-reactive B- cells. B-cells that do not receive a survival signal die. C. Evolution of broadly neutralizing antibodies Theoretical work [250, 251] that was later verified by experiments has shown [252] show that a single cycle of Many pathogens such as viruses (influenza, VIH) come passing through the light and dark zones is not enough in a variety of strains. Typically, antibodies mature to to generate the numbers of observed high affinity B-cells. recognize just one type of strain, by targeting easily ac- This leads to a hypothesis called recycling where ∼ 90% cessible epitopes on the surface of its proteins. How- of B-cells go back to the dark zone. At the end of the ever, viruses can often easily escape immunity afforded affinity maturation process, B-cells increase their affin- by these antibodies by mutating these epitopes, with lit- ity to the antigen by ∼ 1000 fold [251, 253] (or even tle fitness cost. Epitopes that are more conserved (i.e. ∼ 10000 in rabbits [254]) and the B-cell population is al- in which mutations carry a significant fitness cost to ways oligoclonal [255], with a large diversity of different the virus) would be better targets, but they are usually B-cell receptors [256]. Interestingly, a germinal center is harder to access by antibodies, as viruses have evolved to binary: either it produces many high-affinity cells, or it hide those conserved regions. To mature antibodies that fails to produce high-affinity B-cells [257]. bind strongly to more than one strain (called broadly The details of the heterogeneity of GC have been neutralizing antibodies — BnAbs), the immune system discovered through so-called brainbow experiments [255, needs to be trained with different antigens from different 258], in which individual B-cells were permanently tagged strains. In the context of a vaccination strategy, a cru- during BCR acquisition with one of 10 possible combina- cial question is in what antigens to present, and in what tions of four different colored fluorescent proteins. This order. technique combined with imaging allowed for multicolor To address this question in the context of HIV, Wang fate mapping of a cell’s progeny since all cells that come et al [260] simulated different temporal immunization from the same ancestral cell have the same color. By in- schemes, by considering three antigens (wildtype HIV jecting cells in germinal centers with different dyes and antigen and 2 mutants) and modelling their interaction tracking their movement in mice, Victora et al [255] have with antibodies using a BCR-antigen binding model such shown that in one lymphnode there are many germinal as described in section II D. The virus was modeled as centers active at the same time, producing different di- consisting of a conserved part, in which mutations are versities of cells. Most GCs probed at both 6 and 15 days deleterious, and a variable part. The three considered showed a lot of different colored clusters, with lower esti- immunization schedules were: all three variants together mates of ∼ 50 clones per GC, going up to hundreds. By (scheme 1), WT with mutant 1 at the same time, fol- delaying tag formation until the cells are in the GC, se- lowed by mutant 2 (scheme 2), WT followed by mutant lection in GCs was shown to keep multicolored clusters, 1 followed by mutant 2 (scheme 3). Running these sim- showing that diversity of lineages is always maintained ulations many times, they looked at the antibodies that even if a certain dominance of one specific lineage (typ- come out of these in silico germinal centers and scaned ically less than 40% of all cells belong to the dominant them against a standard panel of antigens that are used lineage) is observed at later times. Notably the hetero- in affinity maturation experiments. The breadth of each geneity was huge: some GC had one lineage making up antibody is defined by how many different test antigens to 80% of cells, others as little as ∼ 20%) two weeks af- it recognizes. Scheme 1 does not produce any broadly ter the infection. The rate of diversity loss was also very neutralizing antibodies (bnAbs). In fact, it hardly pro- heterogenous: some GCs converged to a single lineage duces any germinal centers that have any output as B within a few days, and others took over two weeks, or cells die quickly during the process. The frustration of never converged. Even in adoptive transfer experiments not being able to discriminate between conserved and (see section V), where high affinity clones were intro- non-conserved residues results to a situation where the 44 antibodies are not really selected since a BCR is likely fections. Interestingly, both Murugan et al [261] and seeing very different antigen in each round of selection. Neu et al [262] experimentally found that the majority Scheme 2 does produce bnAbs, but with very low prob- of SHM do not necessarily improve antigen binding, and ability. This is likely what happens during a normal in- that both routes can lead to equally high affinity mu- fection because you get infected with a single strain that tants, which supports the heterogeneity observed by Tas later diversifies. In this case, the affinity maturation pro- et al [255] in the brainbow experiments. Yet, it is still cess in the first round does produce binders to the con- not clear why some individuals fail to produce any good served residues, breaking the frustration. Finally, scheme binding antibodies to the very strong antigenic challenge 3 produces bnAbs with high probability (∼ 69%). The of malaria. sequential application allows the evolving BCR to focus Additionally, the BCR repertoire changes with age, on the only non-moving part of the target — the con- with both positive and negative selection acting differ- served residues. These predictions have been tested ex- ently in older people [263], producing longer CDR3s that perimentally, showing that mice that were sequentially are more promiscious and likely to bind self-proteins. immunized focus their immune response. The rate of observed BCRs with SHM also changes from There are effectively two parameters that govern the infancy to adulthood [264]. In influenza vaccine stud- B-cell germinal center distribution: the probability of ies [262], people born before 1977 show less inter clonal diversity but achieve the same affinity as people born af- surviving selection Ps and the probability of mutation ter 1977. Adaptation via the SHM route is less likely in µ. With probability Ps(1 − µ) the existing B-cells will older people, and they are more likely to use their larger expand, with probability Psµ it will mutate, and with existing memory pools than younger people whose anti- probability 1 − Ps is will die. Wang et al [260] suggest this system is optimally frustrated and the probability of bodies acquire a lot of SHM. As a result, the antibodies survival puts the system in a special regime: too strin- of the older group target a conserved part (the stalk) of the influenza protein as do BnAbs, whereas younger gent selection (small Ps will kill all cells), while too le- people’s antibodies target the head (aiming for a strain- nient selection (large Ps) will fill the germinal centers with existing cells, since µ is small. So only intermediate specific antibody). Younger people can produce BnAbs when hit with a strong challenge (vaccine combined with levels of Ps lead to mutant cells surviving, allowing for many rounds of selection that results in acquiring many a pandemic), as predicted by the analysis of Wang et mutations that give large breadth. Support for this inter- al [260], but the response to subsequent infections is still mediate level selection comes from simulations that show specific. In summary, these results highlight the com- that intermediate antigen numbers result in the largest plexity of BCR maturation, which involves both naive number of mutations in a BCR. However experimental cells (with no SHM) and highly mutated cells. The de- validation is still lacking. tails of the response depend on the dose concentration and timing, past infections, age and antigen complexity. Repeated antigen exposure helps increase the affinity of cells through selection, as in Darwinian evolution. In addition, new naive cells are recruited, adding novelty in the selection process other than through mutation. Mu- D. Population genetics approaches to affinity maturation rugan et al [261] compared the impact of selecting naive clones to that of producing new mutants by SHM. Look- ing at the response of BCR to the malaria parasite in hu- 1. Evolutionary analysis of repertoire dynamics mans coupled with string models of affinity maturaion, they showed that antigens with low complexity are much Recent RepSeq data was used to quantify the evolu- more efficient in generating good binders by means of tionary regime of in-host HIV evolution [265] and in re- SHM affinity maturation compared to high complexity sponse to influenza vaccines [266]. The flu vaccine is antigens, which rely on recruitment of new naive cells. a model of an acute infection, which occurs on short Both routes (SHM and selecting naive cells) where ob- timescales. BCRs from 9 timepoint blood samples, be- served experimentally. Influenza vaccine studies show fore and after the infection over the course of ∼ 2 weeks, that, upon secondary immunization (booster vaccines), resulted in very skewed lineage trees and a U-shaped site- ∼ 7% of BCR contained no SHM [262] (although these frequency spectrum (SFS). SFS corresponds to the dis- experiments showed strong clonal dominance with ∼ 10% tribution of frequencies of mutations in a population, and of clones representing ∼ 90% of sequences [259]). The its shape is often informative about the underlying evo- model of Murugan et al [261] predicts that long expo- lutionary process (see section XII C 8). U-shaped SFS, sure to small amounts of antigen leads to selecting for as observed in BCR lineages after vaccination, are char- antibodies with a lot of SHM, whereas short exposure acteristic of strong selection. However, 3 out 5 of the to high antigen concentrations results in selection on ex- strongly selected BCRs recognized neither the flu vaccine isting naive cell variation, which is confirmed in exper- epitope nor the full virus, suggesting bystander evolution, iments. These results, which hold for relatively short by which non-responsive clonotypes expand in response timescales (∼ few months), are consistent with the ar- to a multitude of signals. This could be a mechanism gument of Wang et al [260] in the case of chronic infor upkeeping memory clones between infections with the 45 same or similar pathogens. derived an effective stochastic equation for the rescaled Skewed trees and U-shaped SFS were also observed in mean binding energy between antibodies and viral epi- the BCR lineages of HIV-infected indivuals [265]. How- topes: ever, the same observations were made in out-of-frame d sequences of healthy individuals. This is not surprising = −2 [θ + θ (N /N )] (79) dt A V A V because the repertoires of healthy people are under con- p stant selection, and out-of-frame sequences evolve in cells +sAσA − sV σV + σA + NA/NV σV ξ, with functional receptors, and their evolution reflects the where σ are the variances of the binding energy across hitchhiking and selection on the whole cell. However, a i the viral and antibody populations, θ , s and N are the finer analysis exploiting the dynamic trends of synony- i i i mutation, selection coefficients and population sizes of mous versus non-synonymous mutations is able to dis- antibodies (i = A) and viruses (i = V ); ξ is Gaussian tinguish selection due to the chronic HIV infection from noise and times is measured in units of antibody coa- overall selection patterns. More interestingly, it finds lescence time. This equation was derived by assuming that the CDR3 region of the BCR of untreated HIV carri- a linear relationship between binding energy and fitness, ers evolves according to a regime known as “clonal inter- and using a additive string model of binding energy as ference” (see section XII C 9). In that regime, several new a function of genotype (see section II D 2). The approx- beneficial mutations arise at similar times and compete imate method to get from the evolutionary dynamics, with each other. This competition slows down adapta- which are defined on the genotype, to an effective equation, as only one the beneficial mutations can survive. tion for the phenotype (binding energy), is described in This analysis is based on estimating the probability that a simpler context in XII C 10. a (beneficial) mutation first rises to a threshold frequency Intuitively, in general mutations on both antibodies x, but then is driven to extinction by a competing mu- and viruses reduce recognition. On the other hand, anti- tation, H(x, x ) = G(0|x)G(x|x ), where G(x|y), called a i i body diversity (σ ) increases binding energy by selecting propagator is the probability of ever reaching x starting A the best binders, while virus diversity (σ ) decreases it at y, and x is the initial frequency of the mutant. Both V i by selecting the best escapers on the viral side of co- H(x, x ) and G(x|x ) can be estimated from the data and i i evolution. The same formalism can be applied to the for various models of evolution. Neutral models cannot evolution of BnAbs, which target epitopes on the con- explain empirical observation, and neither does a sim- served regions of the virus. In that context, the equa- ple model of selection with a fitness advantage. Instead, tions simplify to θ = 0 and σ = 0, because the viral data can be fit by a model of varying selection (see sec- V V epitope is constant. tion XII C 5), where the fitness advantage fluctuates as However, tracking the binding energy is not alone a competing mutants come and go. good signature of co-evolution. Time-shifted statistics, Analysis of phylogentic trees build from RepSeq data such as the time delayed viral fitness (see Fig. 15), can from HIV patients combined with S5F hypermutation be used to identify strongly co-evolving populations. The models [240] also led to identifying the constraints on time delayed viral fitness is defined by the interaction BCR adaptation [267]. Ancestral sequences in lineages between the viral population at time t, with distribu- are more likely to mutate their CDR3s than the frame- tion yγ (t) of genotypes γ, and a population of antibodies work (FWR). The propensity of a given residue to mutate α taken at a later time t + τ distributed according to was decreased more in framework (FWR) regions than in xα(t+τ): F (t) ∼ −s (t) P E yγ (t)xα(t+τ), where CDR3s, but in both cases a decrease in mutability was V,τ V αγ αγ E is the binding energy from the string model between much more likely than an increase. Although most of the α γ antibody α and viral epitope γ. Note that same-time fit- constraints on the residue to mutate in CDR3 that are ness F is equivalent to − up to a selectivity constant. under strong positive selection to increase their binding V,0 If the selective effects on the phenotype are compara- affinity, come from nonsynonymous mutations, in all re- ble in the two populations, s θ ≈ s θ , the current gions up to 21% of loss in residue mutability was caused A A V V virus has highest fitness against antibodies from the near by synonymous mutations, which are the result of neu- past since it has acquired mutations to escape them, and tral evolution. These results also point to a slow down in smaller fitness against antibodies from the future, which adaptation from clonal interference, as expected in the have caught up with these escape mutations (Fig. 15). clonal interference regime. For long times, mutations randomize the genome and Fτ (t → ∞) = 0. If the selection pressure of viruses is larger than that of the antibody sV θV sAθA, the virus 2. Models of co-evolution of phenotypic traits is effectively evolving against a neutral antibody population. It still has a high fitness against previously seen The fitness function for the co-evolution between HIV antibodies, and genome randomization decreases its fit- and BCR depends on the ability of the BCRs to recog- ness advantage against future antibodies, but without a nize the virus, which is a (possibly nonlinear) function of penalty. the binding affinity. By focusing on a single antibody lin- The left slope of the time-shifted viral fitness quanti- eage and a viral population, Nourmohammad et al [268] fies the adaptation of the viral population to the exist- 46

the dynamics of quantitative traits, Nourmohammad et al [268] considered the competition of different lineages during affinity maturation. The change in the frequency of a given antibody lineage (each composed of many geno- C c P α types α) of size NA in the population ρ = α∈C x is C driven by its mean fitness FA compared to the mean fit- P C C ness of antibody lineages FA = C FA ρ :

s C C C dρ C C ρ (1 − ρ ) = FA − FA ρ + ξC , (80) dt NA

where ξC is a Gaussian white noise. The lineage fitness is α α an average of the genotype A fitnesses fC (t) that make up the lineage C, weighted by their frequencies in that α C P α α lineage xC (t), FA = α fC (t)xC (t). The mean fitness of a lineage depends on the genotypes within this lineage γ α and also on the frequency of virus y through fC (t). C The probability of fixation Pfix of lineage C is not only a function of the mean fitness advantage and population size, as in Eq. 114 (Sec. XII C 4), but also of the ability of the population within the linage to adapt. Its value is FIG. 15. Time delayed viral fitness. Adapted from Nour- given by: mohammad et al [268]. Viruses have a positve fitness against antibodies from the past, and a negative fitness again anti- C 0 C P /P ≈ 1 + hNA F (t = 0) − FA(t = 0) i + (81) bodies from the future. The slope of the current viral fit- fix fix A 2 ness coming from past antibodies is equal to the fitness flux NA C hφA(t = 0) − φA(t = 0)i φV (t) – a measure of adaptation to existing antibody pop- 3 ulations, and the slope of the current viral fitness coming C −NANV h|TV →A(t = 0)| − |TV →A(t = 0)|i, from future antibodies is equal to the transfer flux TA→V (t) that describes the pressure on the viral population from the 0 C adaptating antibody population. The blue line describes a where Pfix = ρ (t = 0) is the neutral fixation probability, C C co-adapting population, sAθA ≈ sV θV , the red line describes and where φA and TV →A are the antibody counterpart a virus evolving effectively without antibody selection pres- of φV and TA→V for a specific lineage, and φA and TV →A sure, sV θV >> sAθA and the green line is a regime of much their average over lineages. The first corrective term is stronger antibody adaptation, sV θ< < sAθA. standard and comes from the mean fitness advantage. The second corrective term accounts for the adaptation of each lineage in response to selection of antibodies for ing antibody population and defines the a “fitness flux” viral recognition within lineages. The third corrective P γ ∂τ FV,τ (t − τ)|τ=0− = φV (t) = γ ∂yγ FV (t)(dy (t)/dt). term corresponds to the adaptation or escape capability The right slope measures the pressure on the viral popu- of the viral population and its effect of the lineage. lation from the adapting antibody population, which de- For typical antibodies targeting a variable region of the fines the transfer flux of fitness from antibodies to viruses virus, a broad viral diversity is detrimental as it allows P α ∂ F | = T (t) = ∂ α F (t)(dx (t))/dt). C τ V,τ τ=0+ A→V α x V the viral population to escape immunity (large TV →A. At stationarity both fluxes sum to zero, and the deriva- For BnAbs, which bind to constant regions of the virus, tive is continuous, as in Fig. 15. this effect disappears. In fact the effect is the opposite: The characteristic S-curve shown for sAθA ≈ sV θV is higher diversity increases the probability of BnAb fix- indicative of two competing populations. Neutralization ation compared to a non-BnAbs antibody, whereas at measurements [269–272] show that viruses are more resis- low viral diversity the probability of fixation of both a tant to past antibodies and more susceptible to future an- non-BnAb and BnAbs is the same. This argument was tibodies, which results in the S-curved time delayed viral proposed to explain why BnAbs show up later in the in- fitness [268]. Interestingly, Blanquart and Gandon [269] fection when the viral diversity is large. also showed that antibodies that evolved in one HIV positive patient are better at targeting the virus found in another person than this person’s current antibodies. This result remains unexplained. IX. POPULATION DYNAMICS OF While this analysis only involves a single antibody PATHOGENS AND HOSTS lineage, in germinal centers multiple lineages can compete with each other. Extending their analysis based on 47

ag A. Viral fitness models where the Fα (t) is due to the antigenic component of the immune pressure exerted by the on the virus, and stability The immune system of a population of hosts defines Fα encodes the fact that mutations that change a fitness landscape F in which viruses evolve. Models the ability of the protein to fold should be detrimental have been developed to describe how this immune pres- to fitness. The fitness due to folding depends on the free sure shapes the evolution of both flu [273] and HIV [274– energy of folding G, assuming a two state thermodynamic 276] at the population level. Viruses are usually taken to model of folding [279]: evolve according to traditional population genetics mod- stability ¯ −1 els. Fα ∼ (1 + exp[(Gα − G/G0)]) , (85) This formalism can be used to draw short term pre- ¯ dictions about the future fate of existing strains, e.g. where G and G0 set the energy scale and are learned from of circulating influenza over the course of a year. Viral data. Similarly, the antigenic component of the fitness is populations are subject to stochastic Wright-Fisher dy- taken to be a sigmoidal function of the binding affinity namics, but once they reach large enough numbers they H: grow exponentially following the fitness function (see sec- ag ¯ −1 tion XII C 3). In that deterministic stage, the size of a Fα ∼ (1 + exp[(Hα − H/H0)]) , (86) given viral strain α, X (t) grows (or decreases) according α ¯ to the fitness it experiences: where again G and G0 set the energy scale and are learned from data. These constants are learned from Xα(t + ∆t) = Xα(t) exp(Fα∆t). (82) data assuming that non-epitope mutations in the virus decrease protein stability, whereas epitope mutations de- Alternatively, one can consider the longer-term crease binding affinity. This approach was applied to prevalance of particular strains of very diverse viruses influenza byLukszaand Lässig[273]. The antigenic com- such as HIV across many hosts, and use that to infer the ponent F ag was inferred from ferret blood titers, in which fitness landscape of the virus. To do this, we can assume sera of blood from ferrets are challenged with differ- that the population reaches an “equilibrium” as the re- ent viral strains and their antibody response measured. sult of many mutations and fixation events across time To gain precision, the growth prediction of Eq. 82 were in the entire population of viruses and hosts. The prob- considered at the level of clades rather than individual ability of finding a given viral strain is then assumed to strains by summing over all strains in each clade. In the follow Boltzmann’s law: next section we will see how this model could be used to predict the upcoming dominant strain of influenza. pα ≈ exp(βFα), (83) where β plays the role of an inverse temperature setting the tension between entropy from mutations, or B. Co-evolution between host and pathogen populations genetic drift, and the fitness advantage. One possibility is to make use of the equilibrium assumption and learn the fitness Hamiltonian directly from existing viral The adaptive immune system of hosts also evolves un- data. This has been done for the Gag envelope protein of der the selective pressure of the antigenic environment, HIV [274] using maximum entropy approaches (see sec- by expanding immune receptors that led to successful tion XII E 1), such that the probability of seeing a given recognition. The antigens and the immune systems of in- HIV strain α defined by its amino acid sequence ~σ, is dividuals in a population engage on an arms race, where −1 P P one forces the evolution of the other. The interaction be- pα = p(~σ) = Z exp( hiσi + i,j Ji,jσiσj), where hi define single-site fields that constrain the probability of tween the immune system and different viruses takes on observing the wildtype amino acid at a given site i, and different forms. While HIV strains undergo very dynamic Jij are interaction terms between amino acids (assuming in-host evolution [270] experiencing many mutations and a binary representation of the virus with σi = 1 denot- strong selection accompanied by clonal interference [280], ing the consensus amino acid and σi = 0 a mutation the influenza virus evolves more slowly, and at scale of at time i). Such models have been inferred from mul- the whole population of hosts, with few mutations in the tiple sequence alignment of the protein, and then used same host. We have discussed HIV evolutionary models to evolve in silico viral proteins in a population of host in section VIII. Here we turn to the effects of population individuals [274, 276], using quasi-species equations in- level co-evolution on immune repertoires, characteristic troduced by Eigen [277, 278], to make predictions about of viruses such as influenza. HIV evolution. This is currently an emerging field. Most of the treat- Instead of learning the fitness directly from the se- ments of immune systems so far have been coarse-grained quence data, another strategy is to derive the fitness of and reduced to the effective selective pressure they exert each strain by decomposing it into two components on pathogens. This pressure is seen on antigenic maps, which place pathogens in a common space according to ag stability Fα(t) = Fα (t) + Fα , (84) the similarity of the immune response to them (measured 48 in the sera of ferrets). In practice, antigenic maps are pro- co-evolution problem is based on Susceptible-Infected- duced using dimensionality reduction algorithms to re- Recovered (SIR) Models. These models have been used duce the response to a two dimensional manifold. These for a long time to look at epidemic spreading within pop- methods have been extremely useful in tracking influenza ulations [120, 284]. Many of these approaches are based evolution [281] and show that single point mutations can on simulating sets of nonlinear equations. Originally result in a completely new response (new faraway clus- these models were used to study slowly evolving viruses, ter), while some multiple mutations do not change the such as measles [284]. However recently, they have been type of response (same cluster). tied with viral fitness models that account for the evo- However, the link between the molecular interaction lution of the virus. We already discussed one type of between immune receptor (both BCR and TCR, as intro- these models, for in-host evolution of HIV in section VII. duced in section II D 2), and the these phenotypic maps Influenza, unlike HIV that evolves within the host or- remains to be explored. An outstanding question is what ganism, evolves mainly at the level of a population. Re- evolutionary constraints these molecular details impose. cent models [285, 286] of influenza evolution combine For example, is the order of mutations important, and are traditional SIR approaches with data-derived knowledge all the mutations independent? Deep mutational scan- about flu strain evolution. Specifically, in Yan et al. [286], ning experimental techniques exist now to both find the the model is of the form given by Eqs. 170-172 for each best binding proteins in multiple rounds of selection ex- antigenic strain a, with the Susceptible equation solved P periments, and to map out the spectrum of possible so- explicitly, Sα ≈ exp(− α0 Kαα0 Rα0 ), where Sα and Rα lutions. are the fractions of individuals who are susceptible and Recently researchers have very successfully predicted recovered from (and thus immune to) strain α, respec- −|α−α0|/d short-term flu evolution [273, 282]. There are two suc- tively. Kαα0 = e defines a cross-reactivity Ker- cessful types of models in this class. The first uses the nel of range d, where distances correspond to the number diversity of existing strains and extrapolates the tree of mutations in an infinite-genome model. This equa- branches that have recently expanded to predict the dom- tion means that individual who are outside of the cross- inant strains in the near future [282]. This approach reactivity range of the recovered individuals are suscep- makes use of the coalescence framework described in sec- tible to be infected by a given strain. tion XII C 7 and since it relies mainly on evolutionary Within this setup, a mutation in the virus introduces 0 properties, can easily be extended to other globally evolv- a new strain α that increases its distance to all existing viruses. The main idea is to statistically infer the ing strains by one. In a perturbative limit, this muta- characteristics of the evolutionary process from the ex- tion increases fitness by a fixed and constant amount −1 isting influenza trees, with no reference to the immune s ∝ d . This situation can be mapped onto a fitness system. The second [273] is based on molecular informa- wave model where many beneficial mutations compete tion about the influenza antigen [281, 283] and identify- against each other [287] (see section XII C 9). In the fit- ing successful mutations. In practice these are stochastic ness wave description, the viral population has a distri- models that assess the probability of future strains. This bution of fitnesses and reproduces according to that fit- method incorporates an effective treatment of the im- ness. The dynamics are dominated by events happening mune system, based on the response of ferret serum to flu at the stochastic nose of the distribution, i.e. strains with strains, as explained in the previous section. In practice the the largest fitness fm. The beneficial mutations that this treatment ranks influenza strains by the strength of govern the future fate of the population all occur at that serum response, but has no information about the molec- stochastic nose, with rate r ∼ fm/ ln(fm/µ), where µ is ular basis of the response, potential overlap for similar the mutation rate. strains and its evolution. It then builds a fitness model, The cumulative effect of these stochastic beneficial where the future frequency of a given viral strain depends mutations at the nose can be described by an effective on its structural stability and ability to escape the im- Langevin equation: mune system. The details of these interaction models dfm sfm are learned from data using advanced statistical infer- = − Itot + sξ(t), (87) dt ln(f /µ) ence techniques. The second method relies much more m on biophysical details and is therefore in principle more where ξ(t) is a Gaussian white noise of amplitude adapted to influenza, although the approach has been ex- hξ(t)ξ(t0)i = rδ(t − t0), and where the negative fitness tended to predict the evolution of tumorous clones [39] as term −Itot comes from the population of immune sys- a function of their immunogenicity score (see Sec. II D 4). tems catching up with the viral strains in the bulk of Despite their differences both approaches are similar in the distribution, decreasing the fitnesses of all existing style: they rely on recent evolutionary traces (from the strains. This Langevin description makes it possible to last couple of months) to predict the dominant strain up estimate the rate of “speciation”, whereby two mutant to a year in advance. From the evolutionary perspec- strains at the nose escape the influence of the current tive both models encode the idea of clonal interference immune systems in different way, creating two indepen- between viral strains (see section XII C 9). dently evolving antigenic niches which separate once their −d/q Another theoretical approach to the viral-immune genetic distance reaches at least ∼ d, rsp ∼ re , where 49 q = fn/s is the typical number of beneficial mutations XI. GLOSSARY that strains at the fitness nose accumulate relative to the rest of the population. These arguments can be used to explain the deep splits Immunology can be painful at time for physicists in flu strain topologies that have been observed in data because of its “jargon”. To help with accessibility, and simulated using similar types of equations [288, 289]. we try not to focus on specific molecules, but some- In other words, the traveling wave picture also explains times we need to. So here we give a brief overview how influenza can constantly escape the immune system of the main players and rules of the game for the im- without continuous accumulation of genetic diversity. mune system, and also other molecules that are men- Rouzine and Rozhnova [285] applied a similar mapping tioned in the text. For an introduction to immunol- onto a fitness wave description, but in a one-dimensional ogy, we strongly recommend reading “Your Amazing antigenic space. They used the predictions of their con- Immune System” courtesy of the European Federation tinuous traveling wave framework to estimate an anti- Immunological Society http://www.oegai.org/oegai/ genic mutation rate of µ ∼ 3 · 10−5 per transmission 2-PDF/AmazingImmuneSystem.pdf [294]. After this first event and predict the cross-immunity distance of d ∼ 15 glance, we recommend the short but illuminating book nucleotide substitutions that agrees well with indepen- “How the immune system works” by physicist Lauren dent estimates. Sompayrac [1]. While these models give general insights and scal- • Adaptive Immune Response – a response of the or- ing laws to understand viral-immune co-evolution, they ganism to specific pathogens that changes in the ignore the specific molecular details and mechanisms lifetime of each individual. It is based on lympho- of immunity. As more data becomes available about cytes (T and B-cells) recognizing pathogens, pro- the specific interactions between immune cells and viral liferating and then keeping a subset of cells called strains, it may be possible to combine these approaches memory cells that are adapted to previously en- with data-driven models of immune repertoires and viral countered pathogens. Recognition involves interac- genomes, and to use them to make specific predictions tions with many components of the innate immune about the fate of particular viral strains and their rela- system (e. g. antigen presenting cells, cytokines) tion to immune repertoires of hosts. and well as with different cells of the adaptive immune system. Adaptive immune systems first appeared in jawed vertebrates. X. DISCUSSION • Innate Immune Response – the innate immune response involves non-specific types of defense, from Despite the length of this review, there are many sub- physical and chemical barriers (e.g. skin, clotting, jects we did not touch upon that definitely fall into the scratching) to cells and molecules that recognize global topic of quantitative immunology. Some of these, non-specific pathogenic patterns, such as proteins like the whole area of transcriptomics [290] applied to im- of the bacterial cell envelope (LPS - lipopolysaccha- munology, are new and while the methodology is quan- rides). The innate immune system is evolutionary titative, the current experiments are only just starting older than the adaptive one and is found in plants, to give quantitative models of the immune system that insects, funghi, as well as vertebrates. can be linked with a physical understanding (although • Cytokines - small protein secreted by leukocytes to things are moving so fast that by the time this review is enforce cell-to-cell communications in the immune published, this sentence may be obsolete). We also did system. Examples of specific cytokines mentioned not go into the methodology of certain analysis or theo- in the text: IL-2 is a key anti-apoptotic cytokine retical approaches that are widely used, such as machine for T cells as well as an activation cytokine for reg- learning [291] or stochastic gene expression and biochem- ulatory T cells; IL-7 is an hematopoietic growth ical regulation [292, 293], because detailed reviews for (anti-apoptosis, pro-proliferation) factor for lym- the physics audience exist for these topics. We refer the phocytes. curious reader to these papers for the necessary background. Lastly, we only briefly mention some amazing • Interleukins (IL): cytokines historically thought to experimental advances, such as imaging [252] since they be produced by leukocytes. are currently in the process of being used to verify quan- • Leukocytes: the white blood cells of the immune titative models. We hope we have managed to give the system. These include lymphocytes (B, NK or T idea of a vibrant and multi-direction field. We also note cells), granulocytes, monocytes and macrophages that we made presentation choices, which were not easy, (see Fig. 9). because many of the presented topics are linked to other ones. These links will surely become better explored in • TCR: T cell receptor, the main receptor on the sur- the coming years as solid experimental quantification and face of T cells that recognizes pMHC as a ligand for validation of theoretical models becomes the norm. T cell activation. 50

• BCR: B cell receptor, the main receptor on the • Germline mutation: genetic alteration that occur surface of Bcells that recognizes membrane-bound in the gamete-producing cells (sperm and eggs) molecules as ligands for B cell activation. • VDJ recombination – a DNA editing process that • Antigen: biomolecules that trigger an adaptive im- creates T- and B-cell receptors (see section VI B for mune responses. Antigens can be proteins that details). are recognized by antibodies (B-cell mediated responses) or short peptides that are loaded onto • Ig class – B-cells can express different types of con- MHC (T-cell mediated responses). stant regions of their receptors coupled to the same variable region. Throughout their lifetime, they • Epitope: part or whole of the antigen that the im- can change (albeit in a specific order) which gene mune receptor bind to. For B cells, part of the they express – this is called class switching. The protein that the B cell receptor bind to. For T expressed constant region gene determines the so- cells, this corresponds to the peptide loaded by the called class of the B-cell and antibody. The differ- MHC. ent genes are aligned in the immunoglobulin locus, and after the B-cell moves on the next class, pre- • APC (antigen presenting cells) – dendritic cells vious classes get deleted. For example the µ and and macrophages. Surveilling cells that internalize δ genes are first in the heavy chain locus and they molecules and cells from tissues and present them lead to the expression of IgM and IgD chains, both on their MHC type II to T-cells. of which are expressed on naive cells. The order of • MHC: major histocompatibility complex; its main the remaining classes for the heavy locus is IgG3, function is to ”present” short peptides, as antigen IgG1, IgA1, IgG2, IgG4, IgE, IgA2. Class switch- for T cells. It comes in three types, but two of them ing is regulated by cytokines, through regulation of are more relevant for the purposes of this review: gene regulation (see sections IV B and V A). type I and type II. Type I MHC are expressed on most cells in the body, presenting random bits of XII. METHODS protein fragments found in that cell – this informs the surveilling cells if the MHC presenting cell is healthy or not. Type II MHC are expressed only This section is devoted to introducing or expanding on specialized cells (antigen presenting cells) and the more technical details of methods relevant to quan- carry information about the cellular environment titative immunology. These methods and concepts may (whether there is an infection in a given tissue). be of different kinds. Some pertain directly to the bio- physics of immunology and the molecular details of bind- • CD8(+) (killer) T-cell – a type of T-cell identified ing between receptors and cognate ligands. Some may be by its CD8 marker involved in interactions with classical tools from other fields, from statistical learning MHC type I presenting cells. These T-cells trig- to population genetics, but which carry some conceptual ger apoptosis (kill by forcing the cells to kill them- similarity with approaches from statistical physics. The selves) the infected cells. presented methods are almost always cited in the context of recent work in physical or quantitative immunol- • CD4(+) (helper) T-cell – a type of T-cell identified ogy. While they are often not established concepts of by its CD4 marker involved in interactions with immunology, we believe that they might play an increas- MHC type II presenting cells that present peptides ing role in its future development, with more and more from antigens they engulfed. These T-cells produce imports from other fields. cytokines and help orchestrate the response of other immune cells (e.g. B-cells, killer T-cells). A. Physical kinetics • pMHC: a complex of a short peptide (p) and MHC that constitutes a ligand for TCR. Depending on the nature of the embedded peptide, a pMHC can 1. Diffusion-limited reaction rate be agonistic (triggering an immune response), an- tagonistic (extinguishing an immune response) or null. Such hierarchy often lines up with self/non- To compute the association rates of 2 of two molecules self discrimination. (Ligand and Receptor), let’s place ourselves in the reference frame of the receptor, so that the receptor is • Somatic (hyper)mutation: genetic alteration ac- immobile, but the ligand diffuses with coefficient D = quired by a cell that occurs in body cells DLigand+DReceptor. The spatial distribution of the ligand (somaplasm) and that can be passed to the progeny concentration CLigand(~r, t) at steady state obeys Fick’s of the mutated cell in the course of cell division; one law: of the driver for antibody maturation in B cells, and for repertoire generation in T cells. ∇2C(~r) = 0, (88) 51

r→∞ with boundary conditons C(~r, t) −−−→ CLigand and 3. The formation of ligand-receptor pairs: equilibrium and C(r = R) = 0, where R = RReceptor + RLigand is the kinetics sum of the receptor and ligand radii (each modeled by a sphere). Because of spherical symmetry, C(~r) = C(r), we can consider only the radial part of the Laplacian in Once association and dissociation rates are avail- spherical coordinates able, estimating the kinetics of formation of the ligand- receptor pairs is done by considering the reaction: ∂ ∂C r2 = 0, ∂r ∂r kon Ligand + Receptor )−−−−* Complex, (91) k and the steady-state solution is: off and integrating the equation: R C(r) = CLigand 1 − . d[Complex] r = k [Ligand][Receptor] (92) dt on The flux of ligands j onto the receptor sphere is Ligand −koff [Complex]. DC R DC j (r = R) = D∇C = ∞ = ∞ , We have three species whose kinetics we are estimating, Ligand r2 R and two conservation laws (for the ligand and for the receptor), hence the reaction coordinate for this reaction and integrating over the sphere surface one can compute is one-dimensional and can be solved analytically. We total flux: call the complex concentration x = [Complex]” I ~ ~ Φcollision = jLigand(r = R)dS (89) dx = kon(Ltotal − x)(Rtotal − x) − koff x, (93) DC dt = 4πR2 Ligand = 4πDC R. R Ligand where Ltotal is the total ligand concetration, free and The total number of collisions per unit time is complexed, and Rtotal is the total receptor concentration. 4π (RLigand + RReceptor)(DLigand +DReceptor)CLigand and In steady state (for t → ∞) we have dx/dt = 0 and the rate of diffusion-limited interactions between recep- (L − x)(R − x) k tors and ligands kcollision = Φcollision/Cligand is: total total off = = KD, (94) x kon kcollision = 4π (RLigand + RReceptor)(DLigand + DReceptor) . (90) which has two formal solutions 1 x = L + R + K (95) ± 2 total total D 2. The rates of dissociation between two biomolecules p 2 ± (Ltotal + Rtotal + KD) − 4LtotalRtotal

of which only x is physical and satisfies x < The physical chemistry of dissociation of two − L ,R . biomolecules can be modelled in two steps: a slow one total total The kinetics of relaxation that corresponds the breakage of chemical bonds between molecules, and a fast one that corresponds to the rushing dx = kon(x − x−)(x − x+), (96) in of the solvent. Estimating the rate of dissociation koff dt thus corresponds to estimating the slow step, which is truly driven by the quantum physics of bond breakage. can also be integrated: The rate is then encapsulated by the frequency of bond −kreactiont 12 −1 x− − ax+e vibration (kBT/ ≈ 10 s ) multiplied by the proba- ~ x(t) = −k t , (97) bility of of successfully dissociating the molecular pair 1 − ae reaction (exp (−∆Goff /(kBT ))). This bond-breakage free energy where the characteristic rate for the reaction is: sums up the contribution of all the bonds (electrostatic p 2 kreaction = kon (Ltotal + Rtotal + KD) − 4LtotalRtotal, interactions, hydrophobic bonds, Van der Waals forces, and a is a constant depending on the initial condition. hydrogen-bonds etc.) holding a molecular complex to- In the limit of a few receptors Rtotal KD,Ltotal, the gether: it could be computed from purely quantum con- expression simplifies to x = LtotalRtotal/(Ltotal + KD) siderations, independently of the context in which the and kreaction = konLtot + koff . Three regimes can be con- pair is considered (solvated in the intercellular medium, sidered: embedded in the plasma membrane, in vacuum etc.). Yet, this remains a tricky proposition as small errors in • When the ligand exists in high concentration, L estimating the bond energies will yield to exponentially- KD and x = Rtotal, the system relaxes and fluctu- inaccurate estimates of koff . ates with a characteristic rate kreaction = konLtotal. 52

• When the ligand is sparse and in limited amount, the cell at concentration [protein] = X is a dose response Ltotal KD, few complexes form, x ≈ of the X itself (e.g. when X is a transcription factor that LtotalRtotal/KD, and the system relaxes and fluc- binds to the promoter region of the gene for X). Then tuates with a characteristic rate koff . the dynamic equation for X is: • When [Ligand] ≈ K , half of the receptors are oc- d[X] [X] D = k − γ[X], (101) cupied and half are free of ligands, x = Rtotal/2; dt [X] + K this is the mid-point of the dose response when one assess receptor occupancy against increasing con- where k is the maximum production rate, K the concen- centrations of ligands. tration of protein X at half-maximum expression, and γ is a degradation rate (we have used the separation of time scales to eliminate the mRNA stage, as in the previous B. Gene regulation paragraph). This equation has one fixed point X = 0 for k/γ < 0, and Xequilibrium = k/γ − K, for k/γ > K. 1. Basic model A simple stability analysis around that non trivial fixed point, taking [X] = Xequilibrium + , d γ2 k In the simplest quantitative model of gene regulation, = K − , (102) dt k γ we can write the deterministic dynamics of gene transcription and translation as: shows negative restoring force (K − k/γ < 0), meaning d that all fluctuations get quenched and the fixed point is [mRNA] = p − γ [mRNA] (98) dt mRNA mRNA stable. d [protein] = pprotein[mRNA] − γprotein[protein], dt 3. Auto-amplification with multiple transcription factors where pi are the production rates and γi the degradation rates of the molecules, for i = protein, mRNA. Since The complexity and relevance of the auto-amplification transcription and degradation rates for mRNA are of- gene regulatory circuit becomes more relevant when the ten larger than translation and degradation rates for the expression of gene X is regulated by two transcription protein, timescale separation leads to factors TF1 and TF2, i.e. both transcription factors need to bind to the promoter region of gene X to elicit its d pmRNA [protein] = pprotein − γprotein[protein]. (99) transcription. We can compute the state of the promoter dt γ mRNA using classical tools of statistical mechanics. The pro- At steady state: moter for gene X can exist in four possible states with the related probabilities: unoccupied promoter (p0), TF1 pmRNApprotein [protein] = . (100) only (p1), TF2 only (p2), and both (p12), with γmRNAγprotein 1 [TF1] [TF2] [TF1][TF2] This straightforward expression can become arbitrarily p0 = , p1 = , p2 = , p12 = 0 0 , complicated. Both the production and degradation rates Z K1 K2 K1K2 of mRNA and proteins of interest can be complex func- (103) tions of the signalling response, and post-translational with the partition function: modifications and regulated degradation can also hap- [TF ] [TF ] [TF ][TF ] Z = 1 + 1 + 2 + 1 2 , (104) pen. K K K0 K0 In simplest case, these rates can be approximated as 1 2 1 2 constants, when adiabatic conditions apply such that sig- where [TF1] and [TF2] denote concentrations of the nalling responses and cytokine production and consump- two transcription factors and K1 = c0 exp(∆G1/RT ), 0 0 tion occur on timescales much shorter or much longer K2 = c0 exp(∆G2/RT ), K1 = c0 exp(∆G1/RT ) and 0 0 then gene regulation. Including positive feedback loops K2 = exp(∆G2/RT ) their equilibrium binding constants produces more interesting dynamical behaviour, espe- related to the free energies of binding individually (non- cially when the reinforcement is through mutltimerized primed) and cooperatively (primed) to the promoter transcription factors that introduce additional nonlinear- binding sites. ities. When the cooperative binding of transcription factors binding to the promoter region is usually thermodynam- 0 0 ically favored, K1 = K1 and K1 K2, the partition 2. Auto-amplification with a single transcription factor 0 0 function simplifies to Z ≈ 1 + [TF1][TF2]/(K1K2). A common case if when [TF1] = [TF2] = [TF ] and A simple feedback loop corresponds to the case when gene X is transcribed upon homodimerization of a tran- 0 0 the rate of transcription of a protein species present in scription factor in its promoter region (then Ki = K ). 53

from population genetics. The material in this section 1 can be found in many evolution and population text- 0.9 books [295, 296]. To describe the evolution of a mutat- 0.8 ing population, one typically considers a background of X+ genetically identical individuals (or cells, in the context 0.7 of lymphocyte population dynamics). Each one of these 0.6 individuals can acquire a mutation with rate µ, which

Degradation 0.5 starts a new mutant subpopulation. Each individual in , both the ancestral and the mutant subpopulations can 0.4 die or reproduce in each generation, and mutants carry- 0.3 ing non-neutral amino acid substitutions have a different Production 0.2 (higher for beneficial and lower for deleterious mutations) X− 0.1 growth rates. Since it is more likely that a mutation will confer a disadvantage than an advantage, most mutants 0 0 0.5 1 1.5 2 2.5 3 are deleterious. But as we shall see, even beneficial mu- X tations are not necessarily destined to succeed and ultimately the fate of most mutants is to die. Out of the FIG. 16. Bistable auto regulation of a gene. Shown are lucky few that establish subpopulations of significant fre- the production and degradation rates of as a function of the quency, some will completely take over the population — protein concentration X, Eq. 106. The intersections of the they will fix. In this section we calculate the probabili- curves define three fixed points: two stable ones (0 and X+), ties for fates of mutant subpopulations in the simplified and one unstable one (X−). case, compared to somatic evolution of immune repertoires, when there are no sources of new clones other The probabilities of the promoter to be unoccupied (tran- than mutations. scribed at a basal level for auto-activation), poff = p0, and to be occupied (transcribed at an enhanced level for 1. Deterministic mutation-selection balance auto-activation), pon = p12 are: 2 [TF ] Let us start with a situation where two subpopula- pon = 2 02 , poff = 1 − pon (105) [TF ] + K tions exist, the ancestral clone of size n2 individuals This form of regulation, which follows a Hill function and a mutant clone of size n1 individuals. The sub- (with Hill coefficient h = 2), introduces a nonlinearity in populations grow (which accounts for both reproduction the production of X that is of critical relevance in cell and death) with rates γ1 and γ2, respectively, where differentiation. For auto-activation by homodimers, the gi = ri − f(n1, n2) is the balance of growth rate ri and dynamic equation for the regulation of X is: a death rate f(n1, n2. Individuals also mutate from the ancestral population to the mutant one with rate µ1 and d[X] [X]2 = k − γ[X], (106) mutate back with rate µ2: dt [X]2 + K2 dn 1 = µ n − µ n + g (n , n )n (107) where k and γ are production and degradation rates for dt 2 2 1 1 1 1 2 1 X respectively. dn 2 = µ n − µ n + g (n , n )n . (108) As in the non cooperative case, this dynamical system dt 1 1 2 2 2 1 2 2 has a single fixed point if k/γ < K:[X] = 0. But for We also assume, as is often the case in population genet- k/γ > K, it has three fixed points at 0, X− and X+. The stability of these solutions can be deduced graph- ics models that the population size is constant, N = n1 + n = const, which sets f(n , n ) = (r n +r n )/N. The ically (see section V A for details): X− is an unstable 2 1 2 1 1 2 2 constant population size constraint means that knowing fixed point and 0 and X+ are stable fixed points. Hence, this simple system of gene regulation will generate two n the size of both subpopulations is completely deter- types of cells: cells that do not express X, and cells that mined by the fraction of individuals in the ancestral pop- express a high level of X. We see many examples of such ulation x = n1/n, which follows: bimodal distributions in expression of transcription fac- dx tors, cytokines and surface markers in cells of the immune = µ2 − (µ1 + µ2)x + sx(1 − x), (109) system (see section V A). dt where the selection coefficient s = r1 − r2 describes how much faster (or slower if s < 0) the ancestral population C. Population dynamics, genetics grows compared to the mutant subpopulation. At steady state, in the absence of mutations µ1 = µ2 = To study the somatic evolution of immune clones and 0, either the ancestral subpopulation fixes, x∗ = 1 for cell types it is useful to summarize some basic results s > 0, or the mutant one x∗ = 0 for s < 0. In the 54

P absence of selection but with mutations, the relative ra- n Pn→mPt(n), where t labels generations. tio of the mutation rates determines the fraction of each Within this model the mean frequency of the subpop- ∗ P subpopulation, x = µ1/(µ1 + µ2). With both mutation ulation does not change with the time n nPn(t) = and selection a mutation-selection balance is established hn(t)i = hn(t + 1)i. Since the binomial distribution (for s < 0) at x∗ = (1/2)(1 + 2µ/|s| − p4(µ/s)2 + 1) for large N becomes Gaussian, the evolutionary trajec- for µ1 = µ2 = µ. When mutations are rare compared to tory of the number of individuals in each subpopulation the fitness advantage, |s| µ, this balance simplifies to size is well described by a random walk: n(t + 1) ≈ x∗ ≈ µ/|s|. n(t) + η(t)pn(t)(1 − n(t)/N), with η(t) normally distributed. The Wright-Fisher model can be generalized to include 2. Genetic drift selection. The probability to pick a given member i of the parent generation depends on its selective advantage In order to observe a situation described by the de- or disadvantage si, as (1/N)(1 + si)/(1 +s ¯), withs ¯ = terministic equations in Eq. 109, the mutant population P (1/N) i si. needs to grow to a sizeable fraction of the population. In the case of two subpopulations, wildtype (of size However, every mutant appears first in only one individ- n2 = N − n) and mutant (of size n1 = n), with selec- ual and undergoes a subsequent random walk of repro- tive (dis-)advantage s for the mutant, Eq. is modified to: duction and death, which means that the most likely fate N m N−m Pn→m = m p (1−p) , with p = (n/N)(1+s)/(1+¯s) is for it go immediately extinct. This makes the effect of ands ¯ = 1 + sn/N. For large populations, this leads to a small number noise coming from finite population sizes, biased random walk: called genetic drift in population genetics, relevant. We s can explore this effect considering a population that pro- n(t) n(t) duces only neutral mutants (meaning all mutants grow n(t+1) ≈ n(t)+sn(t) 1 − +η(t) n(t) 1 − , N N at the same rate as the ancestor), and keeping the population size fixed to N individuals. If we focus on one (111) individual at some initial time, and follow its offspring, whose determistic part reproduces Eq. 109 in the absence at very long times only two outcome are possible. Either of mutations. its offspring have taken over the whole population, or the lineage has gone completely extinct. Since we start with N individuals, the probability of taking over is 1/N. 4. Probability of extinction The argument generalizes to a subpopulation of size n: the probability that one of its members has taken over We can calculate the probability of extinction of the the population is n/N. mutant subpopulation in the setup of two populations we have described until now. This limit corresponds to a small mutation rate: each subpopulation can fix or go 3. Wright-Fisher model extinct before a new mutation appears. While many immunological situations (e.g. affinity maturation) may not While we cannot calculate deterministically the fate be in that regime, it is an important result to know. The of any particular mutant individual, we can calculate probability that a subpopulation of size n goes extinct, the probability of the evolution of the ancestral and mu- q(n), requires knowing the probability that all their off- tant fractions, assuming two subpopulations and a con- spring, calculated within the Wright-Fisher model, will stant population size as we did in subsection XII C 1. go extinct: Two models, the Wright-Fisher and the Moran mod- N els, each with slightly different setups, describe the neu- X N q(n) = pm(1 − p)N−mq(i), (112) tral evolution of populations. The Wright-Fisher model m assumes discrete, non-overlapping generations: at each m=0 generation, individuals from the previous generation are given a selective advantage s for the population. For cleared, a new sample of N individuals is drawn, each new large N the binomial distribution is peaked, and we can individual picking an ancestor from the previous genera- Taylor expand q(i) around n, q(i) ≈ q(n) + (i − n)q0(n) + tion with probability 1/N. Following a subpopulation of 1 (i − n)2q00(n), which results in: size n in the parent generation, the probability that their 2 offspring comprise m individuals is given by the transi- n 1 n tion probability: 0 = sn 1 − q0(n) + n 1 − q00(n). (113) N 2 N N n m n N−m Pn→m = 1 − . (110) Solving for the extinction probability with boundary con- m N N ditions q(n = 0) = 1 (an extinct subpopulation remains The probability distribution for the size of that subpop- extinct) and q(n = N) = 0 (a fixed population cannot 1−e−sn ulation evolves according to the recursion: Pt+1(m) = go extinct) yields q(n) = 1 − 1−e−sN . The probability of 55

ﬁxation is then: derivation and contribute in a mean “drift”:

1 − e−sn ∂p(x, t) 1 ∂2 P = 1 − q(n) = . (114) = [x(1 − x)p(x, t)] fix 1 − e−sN ∂t 2N ∂x2 ∂ − [(sx(1 − x) + µ − (µ + µ )x) p(x, t)] . If s = 0 we recover the result in subsection XII C 2. ∂x 2 1 2 In the limit of strong positive selection (Ns 1, but (117) s 1) the probability that a founder mutant (n = 1) fixes is proportional to the selection strength Pfix ≈ s, if This Fokker-Planck equation is the stochastic version of the initial population is one individual, or more generally 109, with effective diffusion coefficient x(1−x)/2N. This Pfix ≈ ks, if the initial population is small k 1/s in- expression is also consistent with the random walk ap- dividuals. On the other hand Pfix ≈ 1 for large founder proximation of the Wright-Fisher model (111), meaning populations k 1/s. This defines a threshold of 1/s that the two models are equivalent in that continuous individuals to ensure the survival of the subpopulation. limit. For weak selection pressures |Ns| 1, the probability Eq. 117 has a general solution in terms of Gegenbauer of fixation is equal to the frequency of individuals in the polynomials. At steady state, assuming µ1 = µ2 = −1 population and is independent of s, Pfix ≈ n/N, making µ for simplicity it takes the form p(x) = Z [x(1 − this regime effectively neutral. Lastly, for strongly dele- x)]Nµ−1eNsx, where Z is a normalization factor. We terious mutations, sN −1, the probability of fixation can reinterpret it as a Boltzmann distribution, p(x) = s(N−n) −1 Nsx is non-zero Pfix ≈ e , but is exponentially small. Z p0(x)e , where each mutant individual gives an Nµ−1 energy gain −s, and p0(x) = [x(1 − x)] is the neutral distribution in the absence of selection. Since Eq. 117 is a one dimensional diffusion equa- 5. Moran model, continuous limit, and time varying selection tion, the probability of fixation is calculated from the backward equation for reaching the absorbing barrier at x = 1. This calculation gives the same result as Eq. 114. The above results can also be obtained considering a To model the fluctuations of the environment, we can diffusion process in the number of individuals in the sub- consider a time varying selection pressure s(t) = s + population. To do this, we use the Moran model, in 0 σ(t) with mean s0 and random white noise fluctuations which at each time step we choose one individual to die hσ(t)σ(t0)i = δ(t − t0)Ω around this mean [297]. The and one to reproduce (it can be the same individual), probability p(y, x, t) that the wildtype, starting at initial ensuring the population size is kept fixed. This implies frequency y at time t = 0, reaches frequency at least x overlapping generations, and the typical generation time by time t is given by the backwards equation: is of order N time steps. Assume a wildtype population of size N − n and a 2 ∂tp(y, x, t) = v(y)∂yp(y, x, t) + D(y)∂y p(y, x, t), (118) mutant population of size n. The probability of having n at time step i is given by: where

n + 1 N − (n + 1) v(y) = s y(1 − y) + Ωy(1 − y)(1 − 2y), (119) p (i) = p (i − 1) (115) 0 n n+1 N N n − 1 N − (n − 1) and where the diffusion term cumulates genetic drift and +p (i − 1) n−1 N N fitness fluctuations: " # n 2 N − n2 1 +p (i − 1) + , D(y) = y(1 − y) + Ωy2(1 − y2). (120) n N N 2N With the boundary conditions p(y ≥ x, x, t) = 1 and where the first term describes killing one dominant allele p(y = 0, x, t) = 0, the probability of ever reaching fre- and reproducing a mutant, the second one reproducing a quency x is: dominant allele and killing a mutant, and the last term describes the two possibilities of killing and reproducing 1 − | 1−y/α+ |λ the same allele. In the limit of large population sizes, p(y, x, t → ∞) = 1−y/α− , (121) rescaling time by the typical generation N, t = i/N, 1 − | 1−x/α+ |λ 1−x/α− so that pn(i) − pn(i − 1) = (1/N)∂tpn(t), and Taylor expanding in x = n/N: 1 h p i where α± = 2 1 ± 1 + 2/NΩ and λ = 2 p ∂p(x, t) 1 ∂ s0/(Ω 1 + 2/NΩ). For x = 1 we recover the fixa- = [x(1 − x)p(x, t)] . (116) ∂t 2N ∂x2 tion probability reported in Takahata et al [297], and for Ω → 0 we recover the result for constant fitness (114), −s0Ny −s0N Mutation and selection can be added following a similar Pfix = (1 − e )/(1 − e ). 56

For more than two genotypes [298], Eq. 117 generalizes Defining the generation function G(z, t) = 1 K−1 P∞ n to an equation for a vector ~x = (x , ..., x ) of linearly n=0 p(1, n, t)z we obtain: K PK−1 α independent genotype frequencies (x = 1 − α x ): ∂G(z) 2 ∂p(~x,t) X n ∂ = 1 + (1 + s)G (z) − (2 + s)G(z), (126) = − mα(~x) + Cαβ(~x)s (~x) + ∂t ∂t ∂xα β α,β which is solved with boundary conditions G(z = 1) = 1 1 ∂2 o and G(z, t ) = z: + Cαβ(~x) p(~x,t), (122) 0 2N ∂xαxβ st where the covariance matrices are: (z − 1)(1 − e ) + zs G(z, t) = st . (127) n −xαxβ if α 6= β (z − 1)(1 − (1 + s)e ) + zs) Cαβ(~x) = , (123) xα(1 − xα) if α = β 1 For s = 0 (no selection), we obtain G(z) = 1− 1+(1−z)−1 , a P β the mutation coefficients m (~x) = β(µβ→αx − whose series expansion yields p(1, n = 0, t) = t/(1 + t) α n−1 n+1 µα→βx ) and the selection coefficient is the relative and p(1, n, t) = t /(1 + t) for n ≥ 1. growth rate of genotype β compared to a reference geno- For arbitrary s the mean number of individuals at time β ref st type sα = f − f . t is hni = ∂zG(z)|z=1 = e .The probability of going extinct at time t is est − 1 6. Branching processes G(z = 0) = p(1, n = 0, t) = , (128) (1 + s)est − 1 Branching processes are useful for tracking the fate which goes to 1 − s for s > 0 at t → ∞. We thus recover of the offspring of a individual through time. We will the result that the fixation probability goes to s, as ob- first introduce it in the context of the Moran model, and tained previously in the strong positive selection limit. In then present its more standard applications. Assuming that limit, by the time the mutant escapes genetic drift, birth (with rate 1 + s) and death (with rate 1) as the for n 1/s, its population size is still small compared to only possibly processes, we can track the evolution of the total population size n N, making the branching the probability of having 1 individual at time 0 and n at process approximation appropriate. time t. This probability satisfies a recursion that can be While we have illustrated branching processes in the obtained by considering the possible events occuring in context of a Moran model of evolution, branching pro- the first time step, between times 0 and dt = 1/N [287, cesses are ubiquitous and used in a variety of contexts. 299]: One of the simplest branching process is defined in dis- 1 1 crete time, where at each step a individual can di- p(1, n, t) = 1 − δ N N n,0 vide with probability p, or not divide with probability 1−p [121, 299, 300]. A recursion relation for the generat- 1 1 + 1 − (1 + s)p(2, n, t − dt)+ (124) ing function of the total number of individuals n in a lin- N N P n eage at time t (past and present), G(z, t) = n p(n, t)z , 1 1 can be written following similar arguments as above: + 1 − 1 − (2 + s) p(1, n, t − dt), N N G(z, t) = zpG2(z, t − 1) + (1 − p)z. (129) The first term corresponds to the lineage going extinct between times 0 and dt — and remaining extinct until t. This recursion equation is solved for t → ∞ by: The second term corresponds to a division of the initial p individual between times t = 0 and dt, and the lineage 1 − 1 − z24p(1 − p) G(z, t → ∞) = , (130) then reaching size n from size 2 in the remaining time, 2pz t − dt. The last term corresponds to no change at all. The next step, which characterizes the branching pro- which has a critical point as p → 1/2 [300], when the cess approach, is to assume that the outcome of a lin- average number of offspring equals 1. Rewriting the gen- eage can be deduced from the outcome of each of the erating function as a series we recover the probability of descendants of the first division, taken independently: the total number of individuals (which must be even) p(2, n, t) ≈ Pn p(1, m, t)p(1, n − m, t), which assumes m=0 k that once one individual gives birth to two, the births 1 Γ(k − 1/2) (4p(1 − p)) P = , (131) in these lineages happen independently. This approxi- 2k 2 Γ(1/2)Γ(k + 1) 2p mation is valid as long as n N. Then the recursion becomes in the continuous time limit: For p < 1/2, Pk is a decaying exponential as expected, −3/2 n and for p = 1/2 it decays as a power-law Pk ∼ k . ∂p(1, n) X = δ +(1+s) p(1, m)p(1, n−m)−(2+s)p(1, n), Note that this class of critical branching processes has ∂t n,0 m=0 been used to explain power laws in the distribution of (125) activity in neural networks [301]. 57

7. Coalescence process the tree and the branch lengths in a neutral process. Once the branch lengths (times between each coalescence As seen when calculating fixation probabilities, it is events) have been determined, neutral mutations are dis- often useful to think about evolutionary processes back- tributed randomly along the branch with some rate, so wards in time. This is the basic idea behind a coalescence that their number on each branch follows a Poisson distri- approach, which can be formalised using branching pro- bution. Specificially, the number of mutations π between cesses in the context of the Wright-Fisher model. Here we two individuals, also called pairwise heterozygocity, is give just some basic intuition about how thinking back given by: in time about the history of coalescing sequences in lin- [2µt]π eages can be useful when studying affinity maturation P (π|t) = e−2µnt (135) processes, or tracing phenotypic lineages. W will present π! the coalescence process in a neutral evolutionary frame- for coalescence time t and per-generation mutation rate work. As before the neutral framework provides us with a µ (the total time is 2t because it adds the two branches null model in the case affinity maturation where selection of length t from the common ancestor). Of course, all of is important, but it may also be useful in immunological this breaks down in the presence of selection. phenotyping. The coalescence approach does not con- Thus, the coalescence probability determines the ge- cern itself with mutations, but simply tracks genealogies. netic diversity within a population. The distribution of Mutations can later be added to an existing genealogy π in the population can be obtained by integrating over (tree). the coalescence time (132), We consider two individuals and ask how long ago they Z ∞ π π shared a common ancestor. If one individual has a given 1 −t/N −2µt [2µt] 1 θ parent, the probability that the second cell has the same P (π) = dt e e = , 0 N π! 1 + θ 1 + θ parent, given there are N cells, is 1/N. The probability (136) that they do not have the same parent is 1 − 1/N. Fol- where θ = 2µN = hπi. Thus, in principle, in the regime lowing this reasoning, the probability that they have the of neutrality, characterizing the distribution of the mean same parent t generations ago, but not during the t − 1 mutational distance should allow us to read off the mu- generations is tation rate.

t−1 −t/N Departures from the Poisson distribution is one of the P [T2 = t] = [1 − 1/N] 1/N ≈ 1/Ne , (132) signatures of selection. In the presence of selection, the distribution of branch lengths depends on the details of where we have expanded for large N and T stands for 2 the type of selection we are studying. If we do think time to mean recent common ancestor (MRCA). The about affinity maturation (where selection plays an im- mean time for two cells to coalesce is simply the mean portant role), we see that cells undergo bursts of selec- expectation time of this distribution tion in the germinal centers, followed by periods outside Z ∞ of the germinal centers. Two cells sampled at the same −tN hT2i = dtt/Ne = N. (133) time from the blood may therefore have very different re- 0 cent histories and using their mutation distance to infer The mean time for two cells to coalescence is equal to the a mean mutation rate would be misleading. population size, in units of generation time. More generally, the probability that k cells do not share the same 8. Site frequency spectra and tree balancing parent is Pdiff = [1 − 1/N] [1 − 2/N] ... [1 − (k − 1)/N] ≈ k −1 1 − 2 N . The probability that at least two cells have the same parent (or coalescence in our backwards pic- Another statistics that is easy to calculate within the k −1 neutral model, and can thus be used as a null model to ture) is then Pc = 1 − Pdiff = 2 N . If N k then the probability that more than two cells share the same compare to data, is the site frequency spectrum (SFS). parent in a single generation can be neglected and we The SFS is the number of individuals in the population will assume that in each generation only two cells will (in our case cells) that have a mutation at a given po- share a parent. The distribution of times until the first sition in the aligned sequence with respect to the domi- coalescence is: nant base pair in the most recent common ancestor, and presents it as a histogram: (n1, n2, ...., nN−1), where n1 t−1 P [1st coalescence at time t] = [1 − Pc] Pc (134) is the number of mutations present in a single cell, n2 is the number number of mutations present in two cells, −P t k 1 − k t/N ≈ P e c = e (2) . c 2 N etc. Sites that are not polymorphic (all have the same base pair) do not contribute to the spectrum as they coin- After a coalescent event, there are k − 1 individuals left, cide with the most recent common ancestor of the whole and the process can be repeated until the whole genealog- population. However a mutation at a site that is close ical tree is reconstructed. The coalescence probability to fixation will contribute to the spectrum, although this completely determines the statistics of the topology of mutation now dominates the population. In absence of 58 information about the most recent common ancestor, in 9. Clonal interference population genetics the mutation is often called with respect to the outgroup. In the case of B-cell receptors, a Another concept from population genetics that has good estimate can be the infered from the best alignment been shown to be relevant in BCR affinity maturation to the V, D, and J germline sequences. is clonal interference, which we discussed in more detail We can estimate the SFS for a neutrally evolving pop- in section VIII. Here we recall the back-of-the envelope ulation of constant size N within the Moran model. In arguments of Desai and Fisher [287] to show on a more the continuous time limit, the mean number of mutations classical example why clonal interference slows down the shared by k individuals evolves according to (for k > 1): rate of adaptation. dnk (k + 1)(N − k − 1) (k − 1)(N − k + 1) Given a population size N and mutation rate µ, a mu- = nk+1 + nk−1 tation occurs in any individual with rate Nµ, so the time dt N N −1 k(N − k) between mutations is (Nµ) . Most of these mutations − 2 n = J − J , go extinct due to genetic drift. In the strong selection N k k−1 k (137) regime, a mutation survives genetic drift with probability s. Thus, when mutations are rare, the average time with for a new mutation to occur and fix is (Nµs)−1, and the rate of adaptation is given by v = Nµs2. Jk = [k(N − k)/N]nk − [(k + 1)(N − k − 1)/N]nk+1 rare (138) However fixation itself may take time, and the above the “current” of mutations across subpopulation sizes. picture breaks down when mutations are no longer rare, The first term corresponds to death event occuring in i.e. when the typical time between succesful mutation be- the subpopulation of size k + 1 carrying the mutation comes smaller or of the same magnitude as the time for of interest, and the second term to birth events in the it to fix. After the subpopulation carrying the mutation subpopulation of size k − 1. New mutation always starts overcomes genetic drift which results in an initial popu- with one individual, so that lation size of 1/s, we can consider that the subpopulation grows deterministically with rate s, n(t) ∼ (1/s)est. The dn 2(N − 2) (N − 1) 1 = n − 2 n + µ, (139) mutation will fix when n(t) = N, which gives the fixa- dt N 2 N 1 tion time tfix = (1/s) ln Ns. Mutation are no longer rare where µ is the mutation rate. At steady state the current when (Nµs)−1 ∼ 1/s ln Ns, or Nµ ∼ (ln Ns)−1. is constant and equal to Jk = J1 = µ/N, because each In that regime, many mutants can co-exist at the same new mutation has a probability 1/N to fix, resulting in time, and new mutations can appear in existing mutants a current µ/N of mutations traveling from from size 1 to before these have time to fix. Mutants existing at the N. This implies: same time will have different growth rates, depending on (k − 1)(N − k + 1) µ the number of mutations they have acquired, even if we n = n − , (140) k k(N − k) k−1 k(N − k) assume for simplicity that all mutations give the same fitness advantage. The whole population can be described which is solved by by a fitness distribution and we can notice that the fate nk = µ/k. (141) of a mutation in the bulk of this distribution (a subpopulation that was created some time ago) is different that The SFS can also be calculated approximately in models at its nose (a recently created subpopulation). The nose with selection, but still with fixed population sizes, so we subpopulations are small and susceptible to genetic drift. do not recall these results here [302]. The bulk subpopulations have more individuals and grow Another feature that can be used to identify selection deterministically, but are subject to nonlinear effects of through departure from the neutral model is how bal- competition between individuals. We can consider these anced lineage trees are. Intuitively, a neutral process does two subpopulations separately and then “stitch” the two not favor adding a new mutation to any of the branches, solutions. hence the resulting trees should be symmetric and bal- The individuals at the nose of the fitness distribution anced. A process with selection will preferentially grow have a fitness advantage, lets call it qs, meaning that they the favoured parts of the tree, resulting in some long have q mutations, each confering an advtantage s, com- branches, and other short ones. In the neutral model, pared to the bulk of the distribution. The time needed the number of leaves n in a sublineage at generation t, for an even fitter individual (with fitness (q + 1)s) to follows the distribution pt(n), which satisfies the recur- R τ appear is given by τ that satifies 0 dtµnnose(t)qs = 1, sion: qst where nnose(t) = 1/(qs)e is the number of individuals n n − 1 p (n) = 1 − p (n) + p (n − 1). (142) at the nose of the fitness distribution, following the same t t − 1 t−1 t − 1 t−1 arguments as for the rare mutation case. In the limit of qsτ The sublineage of interest can add one leave by reproduc- e 1, one gets: ing, with probability proportional to its size n, (second τ = 1/(qs) ln (qs/µ). (143) term), or not (first term). Solving by recursion gives a uniform branch length distribution pt(n) = 1/(n − 1). Every time that a new fitter class is added to the distri- 59 bution, the mean of the distribution also increases. After the phenotypic level and we often do not have access to qτ, the old nose of the distribution will become the mean the genotype–phenotype map. Given a fraction xα of (note that the individuals that are less fit than the mean the population carrying allele α, the population mean go extinct deterministically). The populations in fitness and variance of a phenotype E is: class q grow as qs, so in class q − 1 they grow as (q − 1)s, N etc. A lineage originating at the nose grows to dominate 1 X Z the population qτ later, while progressively losing its fit- Γ = hEi = Eαxα ≈ dEEw(E) (148) N ness advantage. By the time it dominates the population, α=1 its relative fitness advantage is zero. On average, it will Z ∆ = hE2i − hEi2 ≈ dE(E − Γ)2w(E), have grown at rate qs/2. This gives the consistency equation (1/qs)eqst/2 = N, with t = qτ. This gives a second estimate of the establishment time: where N is the population size, and w(E) is the distri- 2 bution of this trait in the population. τ = ln Nqs. (144) q2s One can derive an effective equation for the joint evolution of the mean phenotype and its variance [298]: Equating Eq. 143 and Eq. 144, we find the average rate at which the population grows: ∂ h ∂ dΓmut dΓsel Q(Γ, ∆, t) = − + (149) 2 ln Nqs ∂t ∂Γ dt dt q = . (145) ln (Nq/µ) ∂ d∆mut d∆sel − + ∂∆ dt dt We can solve this implicit equation approximately assum- 1 ∂2 ∂2 i ing that q is not too big and neglecting the ln q terms: + CΓΓ + C∆∆ Q(Γ, ∆, t), 2N ∂Γ2 ∂∆2 2 ln Ns q = . (146) ln (N/µ) where we calculate the drift and diffusion terms below. The rate of adaptation in the regime of multiple compet- First lets focus on the diffusion terms, which corre- ing mutations is then: spond to genetic drift are are independent of selection: 2 X ∂Γ ∂Γ s s ln Ns CΓΓ = Cαβ (150) vCI = = . (147) ∂xα ∂xβ τ ln s/µ α,β X 2 α α X α β This rate scales with the logarithm of the population = Eαx (1 − x ) − EαEβx x size, vCI ∼ ln N, much slower than in the rare muta- α α6=β tion regime v ∼ N. The clonal interference regime rare = hE2i − hEi2 = ∆, is not only a regime in which there are many competing mutations, but a regime where new mutations arise where we have used the definitions of the covariance ma- on the background of still relatively low frequency mu- trices in high dimensional space from Eq. 123. Similarly tations, forming competing lineages. The distinction between competition of different clones and clonal interfer- X ∂∆ ∂∆ ence is especially important in affinity maturation, where C∆∆ = Cαβ = h(E − Γ)4i − ∆2 ≈ 2∆2, ∂xα ∂xβ the competition in germinal centers could be between α,β very different clones (which is not clonal interference), or (151) between clones with similar histories, in which case it is where in the last step we have assumed w(E) is ap- clonal interference. proximately Gaussian. A similar calculation shows that C∆Γ = h(E − Γ)3i ≈ 0 with the Gaussian assumption. To compute the drift terms, which arise from selec- 10. Quantitative traits tive effects, we need to specificy how fitness and phenotype are related. For simplicity, let us assume that While models of population genetics are often defined fitness of each individual is a quadratic function of the in the space of genotypes or alleles, what is often mea- phenotype (or expand fitness close to its peak at E∗), ∗ 2 sured in the resulting phenotype, which is a (possibly f(E) = −c0(E − E ) , where c0 is a prefactor that mea- nonlinear) function of the genotype. For instance, for sures the width of the fitness peak. The mean fitness is lymphocyte receptors, the relevant phenotype may be de- then: fined as the binding affinity to epitopes of interest. A projection of a description in phenotypic space, N N X α α X α ∗ 2 α where mutations occur independently at all loci i along F (Γ, ∆) = f x = −c0 (E − E ) x (152) the genome, results in an effective description in pheno- α α ∗ 2 typic space, which is useful because selection occurs at = −c0 ∆ + (Γ − E ) . 60

The change in the mean phenotype Γ due to selection is: where the first term describes mutations, the second term selection and ∂ F = −2c (Γ − E∗(t)) and ξ is dΓsel d Z Z dw(E) Γ 0 Γ = dEEw(E) = dEE( ) (153) a normalized Gaussian white noise. E∗(t) can be a dt dt dt time dependent moving fitness maximum. An analo- Z = dEE (f(E) − F ) w(E) gous equation holds for ∆. Assuming that ∆ changes on much faster timescales than Γ, it can be replaced ∗ 2 ∗ 2 = −c0 (E − E ) − ∆ − (Γ − E ) w(E) by its mean, and the steady state solution has a Boltz- −1 −2NF (Γ) ∗ mann form Qeq(t) = Z Q0(Γ)e , where as before = −2c0∆(Γ − E ). 2 h 1 (Γ−Γ0) i sel Q0(Γ) ∼ exp − 2 h∆i/(4N) ) is the distribution with no Since (by analogy with Eq. 122) dΓ = CΓΓs , using dt Γ selection. Eq. 150 we can verify that the selection coefficient is sΓ = In a time dependent environment the peak of the distri- ∂ F (Γ, ∆) = −2c (Γ − E∗). ∂Γ 0 bution also changes with time according to a prescribed In principle we can calculate the change in the mean model. The population tries to track the fitness peak, d∆sel variance due to selection in the same way dt = without really ever reaching it. From the physical point d R 2 dt dE(E − Γ) w(E) assuming a peaked phenotype dis- of view, the system is maintained out of equilibrium. In a tribution, but in practice its easier to use the analogy changing fitness landscape, where the population history with Eq. 122 and Eq. 151: is described by a sequence of phenotypic trait measure- sel ment (Γ0, ....ΓM ) over time (t0, ..., tM ) the fitness flux of d∆ ∂ 2 = F (Γ, ∆) ≈ −2c0∆ . (154) a population history Φ [303] describes a cumulative se- dt ∂Γ lective effect of phenotypic trait changes: To consider the changes of Γ and ∆ due to mutations M we have to write down a more detailed genotype model X since mutations act on base pairs. If each locus can take Φ = δΓi∂Γi F (Γi, ti) 6= F (ΓM , tM )−F (Γ0, t0). (159) a WT value (σi = 0) or mutant value (σi = 1), each lin- i=1 early additive phenotype (for simplicity) E (for example The lack of equality holds in general because F also has α PL α binding energy) can be written as E = i Ei σi, where an explicit time dependency. One can show that this dis- α Ei is a given sites contribution to the overall phenotype. crepancy is formally equivalent to dissipation in thermal P α α α We have Γ = (1/N) i,α Ei σi x and: systems, and can be related to the entropy production: mut α dΓ 1 X ∂Γ dσ T = i = (155) h2NΦi = KL(P|P ) + boundary terms, (160) dt N ∂σα dt i,α i where PT is the probability of the forward trajectory and X α α α T = − x Ei µ(2σi − 1) = −2µ(Γ − Γ0), P of the backward trajectory and the Kullback-Leibler α,i divergence is defined in Eq. 227. This dissipation has P i been called “fitness flux” in the context of population where Γ0 = (1/2) i E is a phenotype average. Similarly, the variance defined in Eq. 148 is also a sum genetics. over sites, and the change in the variance due to muta- These kinds of phenotypic trait models have been used tions is: as starting points for studying co-evolution as we describe in section VIII. The main difference is that the d∆mut X ∂∆ dσα ∗ = i ≈ −4µ(∆ − E2) (156) fitness objective E is itself a function of the composi- dt ∂σα dt 0 tion of the population with which the initial population i,α i evolves. That other population is also subject to selec- 2 P 2 where E0 = (1/4) i Ei . tion and drift and evolves stochastically, giving rise to Now we have all the elements, the effective diffusion coupled equations [268]. equation (149) in phenotypic space reads: ∂Q(Γ, ∆, t) 1 h ∂2 ∂2 i = ∆ + 2∆2 Q(Γ, ∆, t) 11. Lineage reconstruction ∂t 2N ∂Γ2 ∂∆2 h ∂ ∗ ∂ 2i Lineage reconstruction is a necessary first step in + 2c0∆ (Γ − E ) + 2c0 ∆ Q(Γ, ∆, t) ∂Γ ∂∆ both BCR evolutionary analysis and phenotypic track- h ∂ ∂ i ing. While many software methods exist to reconstruct + 2µ (Γ − Γ ) + 4µ ∆2 − E2 Q(Γ, ∆, t). ∂Γ 0 ∂∆ 0 lineages for evolutionary problems, they are not always (157) well adapted for immunological data. Nevertheless they are often used, and more adapted methods are usually The stochastic equation for the evolution of the mean built upon classical ones, so here we provide a general phenotype can then be read off as: overview of these existing approaches. r Γ ∆ The first problem in reconstructing lineages requires = −2µ (Γ − Γ ) + ∆∂ F + ξ , (158) ∂t 0 Γ N Γ taking the sequence data and identifying clusters of se- 61 quences that share a common ancestor, and therefore be- clusters often have hundreds of leaves. long to the same lineage. A classical strategy to cluster There are two ways of determining distance between datapoints is single-linkage clustering, which builds hi- two node sequences along an edge of a tree: one is to erarchical clusters by iteratively merging pairs together calculate the Hamming distance by simply counting the [304]. The Partis software [180] uses a likelihood ra- number of single nucleotide differences between two se- tio test to decide if a give set of sequences σ1, σ2, ..., σN quences hj, the other is to use the estimated time be- can be grouped as descending from one ancestor or not. tween two mutations tj. The number of mutations in a Specifically Partis uses a Hidden Markov Model (HMM) given time tj is distributed according to a Poisson dis- µtj −µtj based method to annotate each nucleotide in a specific tribution of mean µtj: P (hj) = e . The branch hj ! BCR sequence as coming from a given a set of hidden length refers to the time between two nodes. The tree states corresponding to V, D or J genomic template, or length is defined as the total number of substitutions, from a set of exponentially distributed non-templated in- P htot = branches hj, and is not equal to the sum of sertions. The sum over paths determines the probability branch lengths. Maximum parsimony methods use Ham- P (σi) for each sequence. The same procedure repeated ming distance to define distances on trees, which requires for a set of N sequences, results in the total probability knowing the identity of all the internal nodes and the of a common scenario for these sequences P (σ1, .., σN ). If N tree topology, whereas maximum likelihood sums over the likelihood ratio P (σ1, .., σN )/Πi=1P (σi) > threshold, possibilities for the sequences on the internal nodes, but one concludes that the sequences originate from a com- requires a mutational model. mon recombination. Partis starts with pairs of sequences, The maximum parsimony approach is easy to formu- keeping the cluster with the largest likelihood ratio, and late: for a given topology, it assigns internal nodes so as then adding new members to the cluster based on the to minimize h . It then selects topologies with mini- same test. In practice, to speed up the algorithm Partis tot mal htot. The parsimony scores differ from 0 (when all does not test all possible pairs but creates initial subsets the leaves are the same) to NL (when all the leaves are of data with low Hamming distances. Joining a cluster different, which essentially means this position carries no is irreversible, which can lead to errors in clustering. information). There are (2N − 5)!! topologies to scan Once a lineage is defined, annotation softwares such so many trees have the same htot, including at its mini- as Partis [179] or IGoR [175] can be used to propose the mum. Therefore there is often a large family of most par- naive root of the tree. simonious trees. Apart from the practical problems, the This problem alone is simpler than finding the whole assumptions behind the evolutionary model in this ap- tree genealogy, known as the topology of the tree or the proach are not clear [305]. In practice, the reconstructed branching pattern, because part of the sequence is tem- trees match the true structure if there is little variation in plated by the V, D, and J genes (see Sec. VI B). There are the internal nodes. The main advantage is that it embod- two main classes of methods for tree topology reconstruc- ies the basic intuition for reconstructing the phylogeny. tion: maximum parsimony and maximum likelihood. Let In the maximum likelihood method, given an over- us first explain why it is practically impossible to exhaus- all lineage age T and tree topology, the likelihood is tively sample all tree topologies, and then explain the given by a specific mutation model by varying the branch differences in the two approaches. lengths tj. Within the independent site assumption, We will use the example of binary trees — trees in the likelihood factorizes over sites and the log-likelihood which each node gives rise to only two branches — since PL ` = r=1 ln `r independently traces the evolution of each essentially all loopless trees can be cast into a binary site r. Thus the following equations apply to a single site, form by adding branches of zero length. Starting from but generalize readily to many sites. The probability of the simplest unrooted tree of two leaves connected by mutating the base pair from xi into xj between nodes i one branch, and adding new leaves one at a time, it is and j, is defined as Pxi,xj (tj). For instance, this proba- simple to convince oneself that in each step we add 1 bility can take the following form, although many others leave, 1 internal node and 2 edges, such that a tree with are possible: N leaves has N − 2 internal nodes and 2N − 3 edges (or branches). Adding the Nth leave (N − 1 → N) adds −µtj −µtj Pxi,xj (tj) = e δxi,xj + (1 − e )πxj , (161) 2N −5 topologies, such that the number of unrooted tree P topologies with N leaves TN , is TN = (2N − 5)TN−1, where πx (with x πx = 1) describes the probability that which can be recursively solved to give TN = (2N − 5)!!. a mutation results in base pair x, and is equal to 1/4 For rooted trees, the number of rooted topologies with N for the unbiased 4-base pair model, and tj is the branch leaves is given by TN+1 in terms of the number of non- length (time between nodes i and j). The first term cor- rooted tree topologies, since it just requires adding a root responds to no mutation, and the second to a mutation to the existing unrooted tree topologies. (including one into the same base pair). This formulation The goal of tree reconstruction is to find the topology guarantees detailed balance: πxPxy(t) = πyPyx(t) for all that is consistent with the data. For a tree with N = 10 t. 6 leaves, that means exploring TN ∼ 10 trees and for N = We can now build the likelihood recursively, assuming 76 50, TN ∼ 10 trees, which is prohibitively large. BCR a fixed root identity 0, with sequences denotes as x0 that 62 are not fixed. First consider a tree made of a root and of the population given a fixed environment [309, 310]: two branches of length t1 and t2 leading to leaves 1 and 2, whose identity x and x is known. The likelihood of 1 X 1 2 Λ(dynamics, environment) = lim ln(Nt), (166) T →∞ T this tree is: t=0 X L = πx0 Px0x1 (t1)Px0x2 (t2). (162) where Nt is the population size at a given time, whose x0 evolution is driven by the dynamics. One needs to spec- ify classes of evolutionary dynamics to consider, and the We can now add an internal node 4 to the tree, descend- class of interactions it has with the environment. The ing from 0 and ancestor of 1 and 2, and a leave 3 de- framework is clearly very general, and reduces to a con- scending from 0: strained optimization problem. The main conceptual dif- X X ference with population genetics models studied in this L = π P (t )P (t )P (t )P (t ) = x0 x0x4 4 x0x3 3 x4x1 1 x4x2 2 section is that populations cannot go extinct, and all x0 x4 possible variants are represented. Populations are in a X X = πx0 Px0x3 (t3) Px0x4 (t4)L4(x4), (163) regime where fast growth governs evolution. Alternative x0 x4 approaches could consider optimal strategies for avoiding extinction [311], by minimizing the probability of extinc- where L4(x0) = Px x (t1)Px x . This recursive form al- 4 1 4 2 tion, Pext, whose form depends on the problem consid- lows for efficient calculation of likelihood by successively ered. joining roots of trees into a new root. Formally, joining root j and k into common parent i: D. Ecological models   " # X X L (x ) = P (t )L (x ) P (t )L (x ) , i i  xixj j j j  xixk k k k Population of lymphocytes interact with other through x x j k signaling, competition for resources, cytokines, or anti- (164) gens. For this reason, concepts and mathematical models Leaves are initialized to L (x ) = 1, and the final results j j from ecology are often useful to describe their dynamics. is given by: X L = πx0 L0(x0). (165) 1. Generalized Lotka–Volterra models x0

Given a fixed topology, we can now maximize the likeli- Ecological generalized Lotka-Volterra models [312– hood L over the set of intermediate branch lengths {tj}, 314] describe the co-habitation of multiple species in which results in a score for each topology. The topol- the same environment and account for their interactions, ogy T with the best best ranking score, is the most both direct (one species needs another to reproduce) and likely topology (similarly to the maximum parsimony ap- indirect (through competition for external resources). In proach). general these models have the form: Existing maximum likelihood methods use the inde- dxi pendent site assumption to decrease computational time. = xi [α(~x,~c) − β(~x,~c)] , (167) Many methods also assume homogeneous mutation rates dt across sites and the reversibility of the evolutionary where xi is the frequency of a given species, ci are exter- process. The GTRGAMMA substitution model of the nal factors such as nutrients α(~x,~c) is the growth rate RAxML software [306] uses gamma-distributed mutation and β(~x,~c) the death rate with ~x = (x1, x2,...) and rates for different base pairs, and IgPhyML [307] encodes ~c = (c1, c2,...). The form of the dependence of the a non-reversible mutation model that effectively accounts growth rate on the different species and external factors for the context dependence of mutations (although in a determines the non-linearity of the problem. This is usu- site-independent setup). ally encoded by an interaction matrix between species. Davidsen and Matsen [308] compared maximum likeli- Models where the growth and death rates are the same hood and maximum parsimony methods for BCR lineage for all species are called neutral. Often one introduces reconstruction. They concluded that an improved and in- a carrying capacity which is a form of non-linearity that formed maximum parsimony method outperforms classic describes competition for resources. maximum likelihood. This formulation of the model is very general and the solution depends on the specific assumptions. In general these models are solved numerically, and the results 12. Population growth rates depend on the numerical values of the parameters that are often not known with great certainty. These mod- Another way to study the evolution of populations is to els are often used to ask questions about the coexistence study dynamics that maximize the long term growth rate of different species, as well as speciation itself — why 63 are ecological environments with different species stable? ing pathogens. Since we often do not have experimental information to dS parametrize the interaction matrix, random matrix mod- = −βIS, (170) els have been succesfully used to effectively describe the dt dI interactions [315]. In this approach one uses the fact = βIS − νI = xI, (171) that within a family of random matrices (i.e. matrices dt dR whose elements are chosen from the same distribution) = νR, (172) the eigenvalues of these matrices are the same. Near dt the fixed point, the stability of the system is explored where β is an effective infection rate, ν is the rate of re- by linearizing the system of equations in Eq. 167, and covery, and ν(βS/ν − 1) = x defines an effective growth the eigenvalues of this linearized matrix determine the rate, or fitness of the host population. Within this for- stability fo the ecosystem: mulation no one can die. Since ∂tS + ∂tI + ∂tR = 0, the total population size is constant, N = S + I + R =const. dxi X −1 = −x + K x (t), (168) The typical recovery time is Tr = ν and the typical dt i ij j j timescale for a infected-susceptible individual interaction −1 is Tinf = β , meaning that an infected individual man- where is the interaction strength and K is a random ages to infect R0 = Tr/Tinf = β/ν individuals before interaction matrix. As a result the stability of the sys- recovering. R0 is also called the infection radius or basic tem does not depend on the realization of the interaction reproduction number. matrix, just on its statistics. For the matrix to be stable, Using Eqs. 170-172, we find S = S(t = 0)e−R0(R−R(t=0). At infinite time (t → ∞, I = 0), the all the eigenvalues of K, λi have to satisfy λi ≤ 1, which number of recovered individuals reads: is fulfilled in the largest eigenvalue satisfies λmax < 1/. In the case of Gaussian random matrices, the properties R = N − S(t = 0)e−R0(R∞−R(t=0)), (173) are determined by their first two moments and a strong ∞ transition√ occurs for N → ∞ where the system is stable which means that at an end of an epidemic there are for < 1/ 2 and otherwise unstable. This result holds still susceptible individuals. From Eq. 171 we find that only if the connectivity scales with the size of the system. S(t = 0)R0 > 1, leads to ∂tI > 0 and an exponen- Incidentlly, random matrix theory also has been used tially growing infection (i.e. epidemic outbreak), whereas when looking at covariances in sequence variation (see S(t = 0)R0 < 1 leads to ∂tI < 0 and quenching of the section XII E 6). The general idea is that beyond the infection. Since these equations are deterministic, the first couple of eigenvalues of the covariance matrix of initial condition determines the infection spread for all amino acid variability (which often stems from phyloge- times. The effective number of susceptible individuals netic bias), eigenvalues are well approximated by a ran- that can propagate the infection determines its future. dom matrix. For R×C Gaussian random matrices, where SIR equations can be extended to account for host the elements Kij are chosen from a Gaussian distribution birth (with rate α) and death (with rate µ), that lead with mean 0 and variance σ, the density of eigenvalues to equations of the form: of the covariance matrix Y = hKKT i/R is given by the dS Marchenko-Pastur distribution [316]: = α − µS − βIS, (174) dt 1 p(θ − x)(x − θ ) dI ρ(x) = + − , (169) = βIS − νI − µI = xI, (175) 2πσ2 θx dt dR √ = νR − µR, (176) 2 dt where θ = C/R and θ± = σ (1± θ). with two stable steady state solution: pathogen free, (S, I, R) = (α/µ, 0, 0); and permanent infection, (S, I, R) = ((ν + µ)/β, (µ/β)(R0 − 1), (γ/β)(R0 − 1)), 2. Susceptible-Infected-Recovered (SIR) Models where R0 = βα/ (µ(µ + ν)) is the infection radius. For R0 ≤ 1, the system evolves towards the pathogen-free Susceptible-Infected-Recovered (SIR) models are used solution, while for R0 ≥ 1 the permanently infected so- to study the spread of epidemics at the population level. lution is reached. An SIR model considers three possible kinds of individu- The basic SIR model can be made more complicated in als: ones that are susceptible to the infection (S), infected many different ways. One other variation called the SIS (I) and recovered (R) and therefore usually immune. The (Susceptible – Infected – Susceptible) model describes a simplest SIR model assumes that the birth-death of the situation where an infected individual does not have long host individuals happens on slower timescales than the lasting immunity. This model is inspired by influenza, spread of the epidemic itself. This model is a good de- where the virus mutates fast enough that a past infec- scription of measles and other infections by slowly evolv- tion does not necessarily guarantee immunity (although 64 current models of influenza account for more detailed de- At steady state, we have: scriptions of cross-reactivity – see section IX). The model just has two states: Θ(1) − Θ(x) G0(x) = , (184) (ν − µx)(1 − x) dS = −βIS + νI, (177) dt where we have used the balance between birth and death dI of clones dG(1)/dt = Θ(1) − νN = 0. Using (Θ(1) − = βIS − νI = xI. (178) 1 dt P PC−1 i Θ(x))/(1 − x) = C>0 θc i=0 x , and expaning 1/(ν − P∞ j j Using S = N −I, we obtain a single differential equation: µx) = (1/ν) j=0(µ/ν) x , we obtain:

dI ( C C−1 2 1 1 ν X µk = (βN − ν)I − βI , (179) N = θ − 1 dt C µ − ν C µ k ν k=1 (185) which has two fixed points I = 0 for R = (β/ν)N < 1 ∞ " C #) 0 X ν and I = (βN −ν)/β for R0 > 1. The second fixed results + θ 1 − , k µ from the solution: k=C (βN − ν) 1 I(t) = . (180) which reduces to (βN−ν) β 1 + ( − 1)e(ν−βN)(t−t0) βI0 θ 1 ν C NC = (186) We note that the SIR equations (Eqs. 170-172) differ ν C µ in its assumptions from the two species Lotka-Volterra equations (see section XII D 1) between and a host and a in the simple case θC = θδC,1. The total number of clones pathogen. Specifically, SIR equations do not assume that reads: everyone can get infected, while Lotka-Volterra equations  C  k 1 − ν do. However, generalized Lotka-Volterra equations can 1 X X µ easily be modified to account for different subclasses of Ntot = G(1) = θk µ − ν  C hosts. Lotka-Volterra equations on the other hand do k C=1 (187) explicitly model the viral population, and allow for host- ∞ C ) µk X ν 1 pathogen oscillations that SIR models do not, and they + − 1 , ν µ C do do not assume a constant population size. Notably, C=k+1 the SIS equations (Eq. 177) are a two-species realization of Lotka-Volterra dynamics, with the additional assump- and tion for constant population size.

In section VII we describe equations that model the Ntot = (θ/ν) ln[1/(1 − ν/µ)] (188) in-host evolution during HIV infections. They are very similar in spirit to SIS equations which explicitly consider for θC = θδC,1. In the limit of rare divisions ν µ, only viral dynamics. clone sizes below the initial clone size, C ≤ k, contribute, so that P P 3. Solution of stochastic population dynamics with a source k θkHk k Hkθk Ntot = = Ctot P , (189) µ k kθk Here we present the calculations leading the results Pk presented in Sec. VII, some of which are original to this where Hk = i=1 1/i are harmonic numbers. In the review. To solve Eq. 62, we deﬁne the generating func- opposite limit of division balancing death, µ − ν µ, tions: the long tail of the second sum dominates: +∞ ln(1/) P kθ X C N = k k = C ln(1/), (190) G(x) = NC x , (181) tot µ tot C=1 +∞ with = (µ − ν)/µ. X C Θ(x) = θC x . (182) Eq. 183 can be solved out of steady state with the C=1 method of characteristics. Deﬁning F (x) = G(x) − G(1) The time evolution of the population is then governed and making the change of variable y = ln[(x − µ/ν)/(x − by: 1)]/(µ − ν), this equation reduces to: dG(x) ∂ ∂ = Θ(x)−µN +(νx2 +µ−(µ+ν)x)G0(x). (183) − F = Θ(x) − Θ(1). (191) dt 1 ∂t ∂y 65

This equation can be solved in absence of a source term 4. Solution to foward jump process with opposing drift and (Θ = 0): F0(x, t) = A(t + y), with A(y) = F0(x, 0), source yielding: Consider the process described by (68) generalized to an arbitrary distribution of jumps: µ/ν − (x − µ/ν)/(x − 1)et(µ−ν) F (x, t) = F , 0 0 0 t(µ−ν) Z ∞ 1 − (x − µ/ν)/(x − 1)e ∂ρ ∂ρ ˜ (192) = µ + dy J(y)[ρ(x − y) − ρ(x)]+θ(x), (197) ∂t ∂x 0 Starting with a single clone of size s at t = 0, G0(x, 0) = s s x and F0(x, 0) = x − 1, the coeﬃcients of the solution where absorbing boundary condition at x = 0. where G0(x, t) to the homogeneous equation (Eq. 191 with Θ = J(y)dy is the rate (per unit time) of jumps of size between 0) give Green’s function, deﬁned as the probability that y and y + dy. This equation can be solved exactly by the clone starting at size s at t = 0 has size C at a later considering the Laplace transform of ρ: P C time t, G0(x, t) = C P (C, t|s, 0)x . Expanding (192) Z ∞ in x, one obtains: ρˆ(k, t) = dx e−kxρ(x, t), (198) 0

rC which satisﬁes: P (C, t|s, 0) = g(C, s, t) = (1 − rz)C+s ∂ρˆ = µ(ρ(0, t) + kρˆ(k, t)) + (Jˆ(k) − Jˆ(0))ˆρ(k, t) + θˆ(k), min(C,s) ∂t X sC − 1 × rn(1 − r−1)2nzn(1 − z)s+C−2n, (199) n n − 1 ˆ n=1 where J(k) andρ ˆ(k) are the Laplace transforms for J (193) and θ˜. The steady state solution is:

θˆ(k) − µρ(0) −t(µ−ν) ρˆ(k) = . (200) with the shorthands z = e , r = ν/µ. The combi- Jˆ(k) − Jˆ(0) + µk natorial factor in the sum comes from counting the number of ways there are to choose s − n factors of order To get a physical solution, a pole at k = 0 cannot exist, so s 0 in x, n , and n factors of order (j1, . . . , jn) so that that the total number of clonesρ ˆ(0) is well deﬁned. This Pn C−1 ˆ i=1 ji = C, n−1 , in a series expansion of the form is satisﬁed for µρ(0) = θ(0), which is simply the condition [a + b((cx) + (cx)2 + (cx)3 + ...)]s = [a + bcx/(1 − cx)]s that the rate of birth of new clones, θˆ(0) be balanced by to which (192) can be reduced. the rate of death, µρ(0). Then solution becomes We can now use Green’s function to calculate the dy- θˆ(k) − θˆ(0) namics of the distribution with an abitrary initial condi- ρˆ(k) = . (201) tion: Jˆ(k) − Jˆ(0) + µk To understand the behaviour of ρ(x) at large x, we have X Z t X to examine poles for negative k, k = −α, which satisfy N (t) = N (0)g(C, k, t)+ dt0 θ (t0)g(C, k, t−t0). C k k the condition: k 0 k (194) Z ∞ dx J(x)(eαx − 1) = µα. (202) In the case of an empty immune system at t = 0, NC (0) = 0 0, and constant thymic output θk, we get: The left-hand side has derivativeν ¯ ≡ R dx J(x)x with

min(C,k) respect to k0, which is the average growth rate of a clone. rC X X kC − 1rn(1 − r−1)2n To guarantee that all clones eventually go extinct, that N (t) = θ C µ − ν k n n − 1 n number must be smaller than µ. Therefore (202) has k n=1 only one solution α > 0. At large x the behaviour is n 1 × [z F1(n, 2n − k − C,C + k, 1 + n, z, rz)]e−t(µ−ν) , dominated by that pole,ρ ˆ(k) ∼ 1/(k+α), yielding ρ(x) ∼ (195) e−αx and: 1 NC ∼ . (203) where F1 is the Appell hypergeometric function of two C1+α variables, and where we made the change of variable 0 The total numbers of clones and cells read: z = e−(t−t )(µ−ν) to carry out the integral in (194). This R ∞ equation simpliﬁes greatly for θk = θδk,1: 0 dx θ(x)x Ntot =ρ ˆ(0) = R ∞ , (204) 0 dx J(x)x − µ ∞ C −t(µ−ν) C R x 1 r 1 − e 0 dx θ(x)(e − 1) N (t) = . (196) Ctot =ρ ˆ(−1) = ∞ . (205) C −t(µ−ν) R x ν C 1 − re 0 dx J(x)(e − 1) − µ 66

where the approximation is only valid at large n. We In the specific case of fixed-size jump, J(x) = spδ(x − recognize Zipf’s law (56), which implies a power law in ∆x), we get back (70) and (71). Eq. 202 simplifies in the clone size distribution according to (55): the limit of many very small jumps, corresponding to J ≡ R ∞ dx J(x) → ∞ while keeping the average effect 1 1 tot 0 N ∼ = . (209) of jumps,ν ¯ < µ, as well as its second moment, Γ = C C1+1/(1−α) Cρ+1 R ∞ 2 0 dx J(x)x , finite, meaning that the average jump size h∆xi = ν/Jtot goes to zero. Expanding (202) at small x, we get: E. Inference

µ − ν¯ α = 2 . (206) Here we introduce basic concepts and methods for Γ constructing models and inferring their parameters from data. These approaches are becoming increasingly im- which gives back (72) in the case of fixed-size jumps. portant as large datasets are being produced by high- throughput methods, from imaging to sequencing. Some of these methods have been used in only a few appli- 5. The Yule process cations in computational immunology, but we anticipate that their usage will become widespread in future studies. The Yule-Simon [317, 318] process is another type of “neutral” process that is characterized by a distribution with power long tails, which in its tail is reminiscent of 1. Probabilistic inference, maximum likelihood and Zipf’s law (see section VII B). The model was initially Bayesian statistics formulated to describe the distribution of biological genera. It was later adapted in network science and is known Often, empirical data originates from a stochastic pro- under the name of preferential attachement, or the rich- cess. The first source of variability, which is present in get-richer model. In general it is a realization of a birth- almost any measurement, is experimental noise, which death process, where the birth coefficient is proportional is random. In many cases related to biological systems, to the number of individuals. In the model, new individ- from gene expression to cell signaling and random recom- uals are added at each time step. With probability α, the bination of DNA, the underlying processes are intrinsi- new individual gets a mutation (or undergoes a specia- cally stochastic, and so are the models that must describe tion event if we think about species), and with probability them. 1 − α the new individual’s type is chosen among all the Let us assume that a model can be encapsulated into a possible ancestors, i.e. proportionaly to the abundance probability distribution for observations P (x|θ), where x of each type. is the empirical data, and θ the set of parameters defining The probability that the rank of an individual is k the model. In many cases, the data can be decomposed results in a power law distribution at large C: into a set of independent datapoints, x = (x1, . . . , xN ), so that the probability distribution factorizes over these N α Γ(C)Γ(ρ + 1) 1 C observations: = ∼ ρ+1 , (207) Ctot 2 − α Γ(C + ρ + 1) C N Y where Γ(x) denotes a gamma function, and ρ = 1/(1−α). P (x|θ) = P (xi|θ). (210) A precise derivation of (207) can be found in [318], but i=1 the power-law behaviour can be understood intuitively in a continous approximation. Assume that the rate of This probability is called the likelihood (of the data given µt division is µ, so that there are Ctot = e individual at the model). Because of its often multiplicative form, it time t, starting from a single individual at time t = 0. is common to consider instead its logarithm, the log- The rate of new emergence of new mutants is αµeµt, so likelihood L(x|θ), which is additive, or its normalized t 0 µt0 µt PN R variant `(x|θ) = (1/N) ln P (xi|θ). One can think that at time t there are n(t) = 0 dt αµe = α(e − i=1 1) new types. The key point is that the abundance of of xi as a sequence, for instance of an immune receptor a mutant is an exponentially increasing function of its or of a pathogen; a fluorescence signal coming from flow 0 age. A mutant that arose at t0 has size C = eµ(1−α)(t−t ) cytometry or single-cell microscopy imaging; an abun- at time t. n(t0) can thus be viewed as the rank of the dance of RNA transcripts for gene expression, or of se- mutant, ordered by increasing frequency. The abundance quencing reads of immune receptors when estimating the of the mutant arising at time t0, of rank n = n(t0) is, at abundance of lymphocyte clones. The models may be time t: statistical models for the occurence of a particular data point, or stochastic dynamical models of the process. eµ(1−α)t 1 A popular way to infer the parameters θ = (θ , . . . , θ ) C = ∼ , (208) 1 K (1 + n/α)(1−α) n1−α of a model is to find those that maximize the likehood of 67 the data: Gaussian noise, the stochastic model reads:

θ∗(x) = arg max P (x|θ). (211) N Y 1 2 2 θ P (x|θ) = √ e−(xi−θ1−θ2ti) /2σ 2 i=1 2πσ This estimate of the parameters is called the maximum " N # 1 1 X likelihood estimator, and it can be shown to be unbiased = exp − (x − θ − θ t )2 and optimal in the limit of large numbers of observation, (2πσ2)N/2 2σ2 i 1 2 i i=1 in the sense that it saturates the Cramer-Rao bound. The (214) Cramer-Rao bound gives a lower bound on the variance of unbiased estimators of a fixed and deterministic pa- where σ is the amplitude of the noise. Examining this rameter. In practice, this implies that the fluctuations of ∗ expression, one can see maximizing the likelihood corre- the estimator around its true value, θ (x) − θ are given N sponds to minimizing the mean squared error, P (x − by a multivariate Gaussian distribution of mean 0 and i=1 i θ − θ t )2, which is the standard way to do linear fit. co-variance I−1/N, with: 1 2 i This argument generalizes to any fit of a parametrized function, assuming Gaussian distributed noise. X ∂ ln P (x|θ) I (θ) = P (x|θ) (212) The second example is estimating frequencies from a ab ∂θ ∂θ x a b finite set of outcomes. For concreteness, let us assume that we want to describe the probability of finding a paris called the Fisher information matrix. ticular nucleotide, x =A,C,G,T, at a particular location When one has information about the process and its in a genomic sequence. The parameters of the model are parameters independently of the data, through a prior the frequencies of A, C, and G, denoted θ1, θ2, θ3, while belief in the distribution of θ, Pprior(θ), this information the frequency of T is 1 − θ1 − θ2 − θ3. The probability of can be combined with the data using Bayes rules: observing a certain set of observations x = (x1, . . . , xN ) is given by: P (x|θ)P (θ) P (θ|x) = prior . (213) nA nC nG nT P (x) P (x|θ) = θ1 θ2 θ3 (1 − θ1 − θ2 − θ3) (215)

This expression defines the posterior distribution of pa- P where nA = i=1 δxi,A and similarly for the other nu- rameters, given the data. For instance, some param- cleotides. Maximizing the likelihood with respect to eters may have been measured with some uncertainty, the parameters yields the intuitive counting estimator: which can be modelled using a Gaussian distribution ∗ θ1 = nA/(nA +nC +nG +nT ), and similarly for the other p 2 2 2 Pprior(θi) = (1/ 2πσi ) exp[−(θi −θi,meas) /σi ], or there nucleotides. Note that this argument holds for sequences exists natural physiological range for them. A uniform of nucleotides rather than single nucleotides, provided prior, Pprior =const, amounts to considering the likeli- that the underlying probabilistic model assumes that the hood alone. Also note that in the limit of large datasets, choices of nucletoides at each position along the sequence N 1, the log-likelihood dominates over the logarithm are independent of each other. of the prior, which becomes negligible. In that case, the posterior over possible parameters, P (θ|x), is given by a Gaussian distribution over θ with mean θ∗ and covariance given by the same Fisher information matrix I−1/N. 2. Model selection This means that the Bayesian fluctuations, which correspond to our uncertainty about the true values of the The maximum likelihood and Bayesian rules outlined parameters, match the fluctuations of the error we make above assume that the class of models, if not its param- when picking the maximum-likelihood estimate. eters, are known, but this is not always the case. For Using Eq. 213, one can either consider the full range of instance, we may have competing hypotheses between acceptable parameter values from the posterior, by e.g. different models, or we may consider models of increas- sampling from it. Alternatively, one can take its maxi- ing complexity. For instance, in the example of linear ∗ mum, θ = arg maxθ P (θ|x), called the maximum a pos- regression, we may want to know whether the data is not 2 teriori estimator. In the limit of large N, that estimator better explained by a quadratic model x = θ1 +θ2t+θ3t . is equivalent to the maximum likelihood estimator. Comparing the likelihood directly always favors models We now turn to two simple examples of inference prob- with more parameters, which can in turn lead to overly lems for which the maximum likelihood estimator recov- complex models with the risk of overfitting the data. We ers an intuitive answer. First consider linear regression. need a way to compare models with different structures Suppose we have data points (x1, . . . , xN ) taken at times and numbers of parameters. (t1, . . . , tN ), which we want to fit with a linear model, The general approach to compare two model A and B, x = θ1 + θ2t. Implicitly, we must assume that the data is described by probabilities P (x|θA,A) and P (σ|θB,B), is noisy, otherwise a fit would not be necessary. Assuming to evaluate the overall probability of the data given each 68 model, by integrating the over the parameters: hidden variables h, and x the visible variables, then the Z likelihood of the data can be expressed as the sum over KA P (x|A) = d θAPA(x|θA)Pprior(θA|A) (216) the hidden variables: Z N KB X Y X P (x|B) = d θBPB(x|θB)Pprior(θB|B). (217) P (x|θ) = P (x, h|θ) = P (xi, hi|θ). (222)

h i=1 hi The relative probabilities of each model is then evaluated by Bayes’s rule: Expectation-Maximization (EM) consists of maximizing P (x|θ) iteratively. Starting with a guess θt, one can write P (x|A)P (A) a pseudo log-likelihood: P (A|x) = prior . (218) P (x|A)Pprior(A) + P (x|B)Pprior(B) N X X L˜(x|θ) = P (h |x , θt) ln P (x , h |θ), (223) In the limit of large N, P (x|θ ) (and the same for i i i i A A i=1 h B) is very peaked around its maximum likelihood value, i with Gaussian ﬂuctuations around that peak given by which is essentially an average of the log-likelihood of the the Fisher Information matrix IA: full model (i.e. including the hidden variables), weighted

∗ ∗ by the posterior distribution of the hidden variable hi ∗ −N(θA−θA)IA(θA−θA)/2 PA(x|θA) ≈ P (x|θA)e . (219) under the current model θt. In an iteration step, one sets ˜ θt+1 = arg maxθ L(x|θ). As this scheme is iterated, θt By the saddle-point approximation we obtain: converges to a maximum of the true likelihood P (x|θ). q Expectation-maximization has notably been used in KA ∗ ∗ P (x|A) ≈ (2π/N) /det(IA)PA(x|θA)Pprior(θA|A). the inference of generation model for antigen-specific (220) lymphocyte receptors (TCR and BCR), in Refs. [169, If we replace this expression into Eq. 216, and focus 170, 178, 188]. on two terms that do not depend on our prior assumptions, we see two terms: the maximized likelihood for ∗ 4. Hidden Markov models each model PA(x|θA), which quantifies the quality of the p K fit, and “parsimony” term (2π/N) A /det(IA), which quantifies the volume of parameters that are consistent Hidden Markov models (HMM) are models with a hid- with each model. As evident for this expression of the den variable h which is the trajectory of a Markov pro- parsimony, the more parameters, the smaller that volume cess, h = (hx)x=1,...,L: will be. L The ratio of probabilities in (216) between A and B Y P (h) = p (h ) w(h |h ). (224) is sometimes called Bayes’s factor, while the ratio of the 1 1 j j−1 parsimony terms is called Occam’s factor. If we expand j=2 the logarithm of (216) in o(ln(N)) using (220), we obtain, The visible variable x, is also a vector of size L, whose up to a −2 factor, the so called Bayesian Information elements xj are called ’emissions’ and are drawn ac- Criterion (BIC): cording to the value of the hidden variable according QL ∗ to the emission probabilities P (x|h) = j=1 ej(xj|hj). − 2 ln P (A|x) ≈ KA ln(N) − 2N`(x|θA) ≡ BIC(A). A key feature of HMM is that high-dimensional sums (221) over h = (h1, . . . , hL) can be performed recursively us- The BIC is a popular score for comparing models, and ing technique which is equivalent to the transfer matrix can be intuitively understood as a correction to the like- technique in statistical physics. For instance, the forward lihood by a term that penalizes large numbers of parame- algorithm allows one to calculate P (x), which requires to ters. In the large N limit, Eq. 218 tells us that the model sum over all possible trajectories of the hidden variable. with the smallest BIC is more likely to be correct. An- This is done by defining zj(hj) = P (x1, . . . , xj, hj) and other popular score is the Akaike information criterion using the recursive equation: (AIC), defined as 2K − 2N`(x|θ∗), which puts a smaller penalty on the number of parameters. X zj(hj) = ej(xj|hj)wj(hj|hj−1)zj−1(hj−1). (225) Applications of model selection criterion have been nu- hj−1 merous in computational immunology, see e.g. [78, 97, 200, 233, 319]. The probability P (x) = P (x1, . . . , xL) is then obtained as P z (h ). A similar trick allows one to calculate hL L L any marginal of the model, e.g. P (xj) or correlation 3. Expectation-Maximization functions. Combining these recursions with EM defines a powerful algorithm for learning the parameters of the Some models are better defined in terms of variables model, θ = {wj(·|·), ej(·|·)}, called the Baum-Welch al- that are not accessible to the observer. If we call those gorithm. 69

HMM have been applied to model BCR and TCR an- where θa are Lagrange multipliers that must be adjusted notation to the germline in various software, e.g. SODA to satisfy the constraints, and Z is a normalization con- [177], Partis [179], and repgenhmm [320]. stant. It turns out that maximizing the likelihood of a dataset (x1, . . . , xN ) under Eq. 229 over the model parameters (θa) is equivalent to satisfying the constraint 5. Information theory over the mean observables. The inference procedure is computationally hard, and is usually performed using a The entropy of a probability distribution P (x) is de- combination of Monte Carlo sampling methods and gra- fined as: dient descent, or mean-field techniques [322]. Often, the observables are taken to be marginals of variables of in- X S[P ] = − P (x) ln P (x). (226) terest, as well as pairwise of higher-order correlations be- x tween them. The resulting models then fall into classes of inverse statistical physics models, such as disordered It quantifies the randomness of the distribution, and is Ising or Potts models [323]. maximum for a uniform distribution (and zero for per- fectly peaked one). For this reason is is often used as a measure of diversity. Unlike all other diversity measures, 7. Machine learning and Neural networks it is additive, meaning that the entropy of a joint distribution of two independent variables (P (x),Q(y)) is given Modern machine learning techniques are increasingly by S[P ] + S[Q]. popular in computational immunology, and are antici- The entropy is the building block for several pated to become even more popular in the near future. information-theoretic measures. The Kullback-Leibler We refer the reader to Ref. [324] for a review of machine divergence or relative entropy, defined as learning methods aimed at physicists. The best established use of deep neural networks in im- X P (x) KL(P kQ) = P (x) ln , (227) munology is for predicting peptide-MHC binding through Q(x) x a set of tools call netMHC [40, 164]. More traditional machine learning approaches have been used to compare is used a distance measure between probability distribu- and characterize immune repertoires [211, 212]. tions (although not a metric in the mathematical sense Current efforts aim at predicting TCR-antigen bind- because of its asymmetry). Another popular measure is ing using machine learning techniques [24–26], although the mutual information between two variables x and the amount of data necessary to obtain truly predictive y: models is probably still insufficient at this stage.

X P (x, y) I(x, y) = S[P (x)]−S[P (x|y)] = P (x, y) ln . P (x)P (y) x,y F. High-throughput repertoire sequencing (228) It quantifies how much the knowledge of y reduces the The variable region of an antigen receptor chain is entropy of x, and vice-versa since it is symmetric in x about 400 bp long. The sequencing challenge is to cap- and y. It is often used as a non-parametric measure of ture the whole region in one read. Modern methods correlations between two variables. of high-throughput genomic or metagenomic sequencing use shotgun sequencing which breaks the genome of interest into short fragments and after sequencing pastes 6. Maximum entropy models the reads together by putting together overlapping reads with the help of a reference guiding template. Since When the underlying mechanisms that give rise to Repertoire Sequencing (RepSeq) focusses on highly vari- the data are not known, it can be useful to define phe- able regions which includes non-templated insertions and nomenological models based on the observables that are deletions, the sequence must be acquired in one read. deemed important. A convenient way to do this is to Read lengths required for TCR sequences are typically infer maximum entropy models [321], which are proba- shorter (∼ 100 − 150bp) since short fragments of V and bilistic models P (x) of maximum entropy S[P ] subject J genes are enough to distinguish different V genes with to the constraint that they agree with the data on a the help of known genomic templates. However, since choice of key mean observables O (x): P P (x)O (x) = B-cells carry a large number of hypermutations in the a x a V gene, sequencing of longer fragments (∼ 600bp) that (1/N) PN O (x ). It can be shown that the distribu- i=1 a i encompass explicitly most of the V gene is necessary. tion takes an exponential form: Different methods have been developed for sequenc- ! ing DNA and mRNA. In general, the protocols start 1 X P (x) = exp θ O (x) , (229) with isolating the mRNA or DNA of the cells of interest Z a a a (BCR, TCR, in subsets of cells of interest sorted using 70

FACS). The mRNA product is then reverse transcribed proach that exploits the statistics of co-occurence of the onto cDNA and barcoded, sometimes with unique molec- rare events partitioning the chains separately into wells ular identifier (UMI), and amplified by Polymerase Chain is used to identify pairs of TCR sequences that come Reaction (PCR) before sequencing. Different techniques from the same cell from bulk DNA sequencing experi- exist for PCR amplification that either add primer spe- ments [329]. The results of these analysis show that the cific sites or multiple primers. Rapid amplification of pairing of alpha-beta chains is largely independent for cDNA ends (RACE) is one of the most common tech- TCR [145, 328], so the results of bulk analysis for diver- niques, which is based on adding a linker with a primer sity hold. However the affinity and functional properties binding site to a conserved region, on the 5’ end of the V of the BCR and TCR need to be estimated at the level gene where no constant template is available (unlike on of the whole receptor. the 3’ end). In DNA sequencing multiple primers that Lastly, novel high-throughput single-cell barcoded target specific regions of the DNA, introducing a primer mRNA technologies [223] are promising to change the specific amplification bias that can be controlled for by field of immune repertoire sequencing, providing paired spiking in known sequences. chain reads in large numbers. These technologies are Protocols based on sequencing directly DNA have the very similar to those used in 5’RACE with UMI. Droplet advantage that the experiments is free of mRNA expres- based platforms allow in principle for large numbers of sion bias. However, currently mRNA based technologies cells (up to 80 000 per chip) to be analyzed at the same are able to reliably report sequence counts thanks to the time, although these numbers are still much smaller than use of UMI barcoding techniques that are still being de- the 106 cells in bulk experiments. Given that clones have veloped for DNA sequencing. The difficulty with using very low frequencies in bulk experiments (possibly down UMIs in the DNA protocol lies in introducing the barcode to a single cell out of 1011), sampling of the order of 104 before the original sequence is amplified. In the mRNA does not guarantee reproducible experiments. Addition- protocole the barcode is introduced during the initial really the cost remains high, at about 1 USD per cell. At verse transcription of mRNA into cDNA for every RNA the moment of writing, the first analyses based on this molecule. The barcoded sequence is then purified and technology are still underway. the whole product is amplified by PCR. For DNA an additional ligation step is needed, in which the DNA is cut close to the region of interest (keeping in mind that the sequenced lengths is short). Ligation is not an efficient reaction. This approach is simple when many gene copies are available in a sample, but this is not the case XIII. ACKNOWLEDGMENTS in immune repertoire sequencing. Alternatively, the barcode could be added in the first PCR cycle. The product then needs to be well purified making show no barcode The authors thank all past and current collabora- carrying primers are left, at the same time making sure tors for the many useful discussions. TM and AMW that the barcoded sequences is not lost due to its low thanks Meriem Bensouda Koraichi, Barbara Bravi, Vic- fraction compared to non-barcoded sequences, which re- tor Chardès,Thomas Dupic, Cosimo Lupo, Carlos Oli- mains technically tricky. vares, Jacopo Marchi, Maria Ruiz Ortega and Natanael The above procedures sequence the chains of TCR or Spisak for their comments on the manuscript and Mikhail BCR repertoire in bulk, leaving no possibility to figure Pogorelyy and Anastasia Minervina for discussions. out the pairing of alpha and beta, or light and heavy GAB thanks Sooraj Achar, Emanuel Salazar Cavazos and chains in the same cell. Naturally, this limits the discus- Van Truong for their comments on the manuscript. This sions of repertoire diversity, and more importantly at- work was partially supported by the European Research tempts to link sequence to a functional phenotype. Re- Council Consolidator Grant n. 724208 [TM & AMW] and cently these limitations have been overcome by the de- the intramural research program of the National Cancer velopment of single cell repseq sequencing, where the Institute [GAB]. We also thank the KITP, Les Houches PCR reaction is performed either in wells [325] or in EdP and IES Cargèseinstitutes for hospitality during droplets [326–328]. Alternatively, a computational ap- workshops on the topics reported in this review.

[1] Sompayrac L (1999) How the immune system works [4] Jerne NK (1955) The natural-selection theory of anti- (Blackwell Scientiﬁc Publications). body formation. Proc Natl Acad Sci U S A 41:849–857. [2] Burnet FM (1957) A Modiﬁcation of Jerne ’ s Theory [5] Hodgkin PD, Heath WR, Baxter AG (2007) The clonal of Antibody Production using the Concept of Clonal selection theory : 50 years since the revolution. Nature Selection. The Australian Journal of Science 20:67–69. immunology 66:424–448. [3] Ehrlich P (1900) On Immunity with Special Reference [6] Nossal GJV, Lederberg J (2019) Antibody Produc- to Cell Life. Roy. Soc. Proc. 66:424–448. tion by Single Cells. 1. The Journal of Immunology 71

182:1231–1232;. [27] Chao DL, Davenport MP, Forrest S, Perelson AS (2005) [7] Billingham RE, Brent L, Medawar PB (1953) Activity The effects of thymic selection on the range of T cell Acquired Tolerance ’ of Foreign Cells. The Journal of cross-reactivity. Eur. J. Immunol. 35:3452–3459. Immunology 172:603 606. [28] Miyazawa S, Jernigan RL (1996) Residue-Residue Po- [8] Billingham RE, Brent L, Medawar PB (1956) Quan- tentials with a Favorable Contact Pair Term and an titative Studies on Tissue Transplantation Immunity. Unfavorable High Packing Density Term, for Simulation III. Actively Acquired Tolerance. Philosophical Trans- and Threading - 1-s2.0-S002228369690114X-main.pdf. J actions of the Royal Society B: Biological Sciences Mol Biol pp 623–644. 239:378–414. [29] Kosmrlj A, Jha AK, Huseby ES, Kardar M, [9] Lifschitz E, Pitaevskii L (1981) Physical Kinetics Chakraborty AK (2008) How the thymus designs (Butterworth-Heinemann). antigen-specific and self-tolerant T cell receptor se- [10] Hager GL, McNally JG, Misteli T (2009) Transcription quences. Proc. Natl. Acad. Sci. U. S. A. 105:16671–6. dynamics. Mol Cell 35:741–53. [30] George JT, Kessler DA, Levine H (2017) Effects of [11] Halford SE, Marko JF (2004) How do site-specific dna- thymic selection on T cell recognition of foreign and binding proteins find their targets? Nucleic Acids Res tumor antigenic peptides. Proc. Natl. Acad. Sci. p 32:3040–52. 201708573. [12] Gorman J, Greene EC (2008) Visualizing one- [31] KoˇsmrljA, Chakraborty AK, Kardar M, Shakhnovich dimensional diffusion of proteins along dna. Nat Struct EI (2009) Thymic selection of T-cell receptors as an Mol Biol 15:768–74. extreme value problem. Phys. Rev. Lett. 103:068103. [13] Slutsky M, Mirny LA (2004) Kinetics of Protein-DNA [32] Butler TC, Kardar M, Chakraborty AK (2013) Quorum Interaction : Facilitated Target Location in Sequence- sensing allows T cells to discriminate between self and Dependent Potential. Biophysical Journal 87:4021– nonself. Proc. Natl. Acad. Sci. U. S. A. 110:11833–8. 4035. [33] Perelson AS, Oster GF (1979) Theoretical studies of [14] Altan-Bonnet G, Germain RN (2005) Modeling t cell clonal selection minimal antibody repertoire size and antigen discrimination based on feedback control of dig- reliability of self non self discrimination. J. Theor. Biol. ital erk responses. PLoS Biol 3:e356. 81:645–670. [15] Feinerman O, Germain RN, Altan-Bonnet G (2008) [34] Detours V, Mehr R, Perelson aS (1999) A quantitative Quantitative challenges in understanding ligand dis- theory of affinity-driven T cell repertoire selection. J. crimination by alphabeta t cells. Mol Immunol 45:619– Theor. Biol. 200:389–403. 31. [35] Detours V, Mehr R, Perelson a (2000) Deriving quan- [16] Mason D (1998) A very high level of crossreactivity is titative constraints on T cell selection from data on the an essential feature of the T- cell receptor. mature T cell repertoire. J Immunol 164:121–128. [17] de Boer RJ, Perelson ASA (1993) How diverse should [36] Wang S, et al. (2015) Manipulating the selection forces the immune system be. Proc. R. Soc. B Biol. Sci. during affinity maturation to generate cross-reactive 252:171. HIV antibodies. Cell 160:785–797. [18] Birnbaum ME, et al. (2014) Deconstructing the peptide- [37] Nourmohammad A, Otwinowski J, Plotkin JB (2015) MHC specificity of t cell recognition. Cell 157:1073– Host-pathogen co-evolution and the emergence of 1087. broadly neutralizing antibodies in chronic infections. pp [19] Adams R, Kinney JB, Mora T, Walczak AM (2016) 19–22. Measuring the sequence-affinity landscape of antibod- [38] Adams RM, Kinney JB, Walczak AM, Mora T (2019) ies with massively parallel titration curves., Technical Epistasis in a Fitness Landscape Defined by Antibody- report. Antigen Binding Free Energy. Cell Syst. pp 1–8. [20] Glanville J, et al. (2017) Identifying specificity groups in [39] Luksza M, et al. (2017) A neoantigen fitness model the T cell receptor repertoire. Nature advance on:94–98. predicts tumour response to checkpoint blockade im- [21] Dash P, et al. (2017) Quantifiable predictive features de- munotherapy. Nature 551:517–520. fine epitope-specific T cell receptor repertoires. Nature [40] Andreatta M, Nielsen M (2015) Gapped sequence align- advance on:89–93. ment using artificial neural networks: Application to the [22] Shugay M, et al. (2018) VDJdb: A curated database MHC class i system. Bioinformatics 32:511–517. of T-cell receptor sequences with known antigen speci- [41] McKeithan TW (1995) Kinetic proofreading in T-cell ficity. Nucleic Acids Res. 46:D419–D427. receptor signal transduction. Proceedings of the Na- [23] Tickotsky N, Sagiv T, Prilusky J, Shifrut E, Friedman N tional Academy of Sciences 92:5042–5046. (2017) McPAS-TCR: A manually curated catalogue of [42] Hopfield JJ (1974) Kinetic proofreading: a new mecha- pathology-associated T cell receptor sequences. Bioin- nism for reducing errors in biosynthetic processes requir- formatics 33:2924–2929. ing high specificity. Proc Natl Acad Sci U S A 71:4135–9. [24] Jurtz VI, et al. (2018) NetTCR: sequence-based predic- [43] Ninio J (1975) Kinetic amplification of enzyme discrim- tion of TCR binding to peptide-MHC complexes using ination. Biochimie 57:587–95. convolutional neural networks. bioRxiv p 433706. [44] Fran¸coisP, Voisinne G, Siggia ED, Altan-bonnet G, Ver- [25] Sidhom JW, Larman HB, Pardoll DM, Baras AS (2018) gassola M (2013) Phenotypic model for early T-cell ac- DeepTCR: a deep learning framework for revealing tivation displaying sensitivity , speci fi city , and antag- structural concepts within TCR Repertoire. bioRxiv p onism. Proc Natl Acad Sci U S A pp E888–E897. 464107. [45] Germain RN (2005) Modeling T Cell Antigen Discrim- [26] Jokinen E, Heinonen M, Huuhtanen J, Mustjoki S, Harri ination Based on Feedback Control of Digital ERK Re- L (2019) TCRGP : Determining epitope specificity of sponses. PLoS biology 3. T cell receptors. pp 4–12. 72

[46] Sykulev Y, Joo M, Vturina I, Tsomides TJ, Eisen HN [65] Cai E, et al. (2017) Visualizing dynamic microvillar (1996) Evidence that a single peptide-mhc complex on a search and stabilization during ligand detection by t target cell can elicit a cytolytic t cell response. Immunity cells. Science 356. 4:565–571. [66] Dustin ML, Chakraborty AK, Shaw AS (2010) Under- [47] Irvine DJ, Purbhoo MA, Krogsgaard M, Davis MM standing the structure and function of the immunologi- (2002) Direct observation of ligand recognition by t cal synapse. Cold Spring Harb Perspect Biol 2:a002311. cells. Nature 419:845–849. [67] Tsourkas PK, Liu W, Das SC, Pierce SK, Raychaudhuri [48] Stefanova I, et al. (2003) Tcr ligand discrimination is S (2012) Discrimination of membrane antigen affinity enforced by competing erk positive and shp-1 negative by b cells requires dominance of kinetic proofreading feedback pathways. Nat Immunol 4:248–54. over serial engagement. Cell Mol Immunol 9:62–74. [49] Lalanne JB, Fran¸coisP (2013) Principles of adaptive [68] Blinov ML, Faeder JR, Goldstein B, Hlavacek WS sorting revealed by in silico evolution. Phys. Rev. Lett. (2006) A network model of early events in epidermal 110:218102. growth factor receptor signaling that accounts for com- [50] Qi SY, Groves JT, Chakraborty AK (2001) Synaptic binatorial complexity. Biosystems 83:136–51. pattern formation during cellular recognition. Proc Natl [69] Lemmon MA, Schlessinger J (2010) Cell signaling by Acad Sci U S A 98:6548–53. receptor tyrosine kinases. Cell 141:1117–34. [51] Huse M, Lillemeier BF, Kuhns MS, Chen DS, Davis MM [70] Yang J, Reth M (2010) The dissociation activation (2006) T cells use two directionally distinct pathways for model of b cell antigen receptor triggering. FEBS Lett cytokine secretion. Nat Immunol 7:247–55. 584:4872–7. [52] Choudhuri K, Wiseman D, Brown MH, Gould K, [71] Fleire SJ, et al. (2006) B cell ligand discrimination van der Merwe PA (2005) T-cell receptor triggering through a spreading and contraction response. Science is critically dependent on the dimensions of its peptide- 312:738–41. mhc ligand. Nature 436:578–82. [72] Fran¸coisP, Hakim V (2004) Design of genetic networks [53] Choudhuri K, van der Merwe PA (2007) Molecular with specified functions by evolution in silico. Proc. mechanisms involved in t cell receptor triggering. Semin Natl. Acad. Sci. U. S. A. 101:580–585. Immunol 19:255–61. [73] Proulx-Giraldeau F, Rademaker TJ, Fran¸coisP (2017) [54] Valitutti S, Dessing M, Aktories K, Gallati H, Lanzavec- Untangling the Hairball: Fitness-Based Asymptotic Re- chia A (1995) Sustained signaling leading to t cell acti- duction of Biological Networks. Biophys. J. 113:1893– vation results from prolonged t cell receptor occupancy. 1906. role of t cell actin cytoskeleton. J Exp Med 181:577–84. [74] Lipniacki T, Hat B, Faeder JR, Hlavacek WS (2008) [55] Kalergis AM, et al. (2001) Efficient t cell activation Stochastic effects and bistability in T cell receptor sig- requires an optimal dwell-time of interaction between naling. J. Theor. Biol. 254:110–122. the tcr and the pmhc complex. Nat Immunol 2:229–34. [75] Altan-Bonnet G, Mukherjee R (2019) Cytokine- [56] Holler PD, et al. (2000) In vitro evolution of a t cell mediated communication: a quantitative appraisal of receptor with high affinity for peptide/mhc. Proc Natl immune complexity. Nat Rev Immunol 19:205–217. Acad Sci U S A 97:5387–92. [76] Vogel RM, Erez A, Altan-Bonnet G (2016) Dichotomy [57] Lever M, et al. (2016) Architecture of a minimal signal- of cellular inhibition by small-molecule inhibitors re- ing pathway explains the t-cell response to a 1 million- vealed by single-cell analysis. Nat Commun 7:12428. fold variation in antigen affinity and dose. Proc Natl [77] Feinerman O, et al. (2010) Single-cell quantification of Acad Sci U S A 113:E6630–E6638. il-2 response by effector and regulatory t cells reveals [58] Posey, A. D. J, et al. (2016) Engineered car t cells tar- critical plasticity in immune response. Mol Syst Biol geting the cancer-associated tn-glycoform of the mem- 6:437. brane mucin muc1 control adenocarcinoma. Immunity [78] Cotari JW, Voisinne G, Dar OE, Karabacak V, Altan- 44:1444–54. Bonnet G (2013) Cell-to-cell variability analysis dissects [59] Schmitt TM, et al. (2017) Generation of higher affinity the plasticity of signaling of common γ chain cytokines t cell receptors by antigen-driven differentiation of pro- in T cells. Sci. Signal. 6:ra17. genitor t cells in vitro. Nat Biotechnol 35:1188–1195. [79] Busse D, et al. (2010) Competing feedback loops shape [60] Valitutti S, Muller S, Cella M, Padovan E, Lanzavecchia il-2 signaling between helper and regulatory t lympho- A (1995) Serial triggering of many t-cell receptors by a cytes in cellular microenvironments. Proc Natl Acad Sci few peptide-mhc complexes. Nature 375:148–51. USA 107:3058–63. [61] Zhu C, Jiang N, Huang J, Zarnitsyna VI, Evavold BD [80] Hofer T, Krichevsky O, Altan-Bonnet G (2012) Com- (2013) Insights from in situ analysis of tcr-pmhc recog- petition for il-2 between regulatory and effector t cells nition: response of an interaction network. Immunol to chisel immune responses. Front Immunol 3:268. Rev 251:49–64. [81] Levine SJ (2004) Mechanisms of soluble cytokine recep- [62] Liu B, Chen W, Evavold BD, Zhu C (2014) Accumu- tor generation. J Immunol 173:5343–8. lation of dynamic catch bonds between tcr and agonist [82] Rose-John S, Neurath MF (2004) Il-6 trans-signaling: peptide-mhc triggers t cell signaling. Cell 157:357–368. the heat is on. Immunity 20:2–4. [63] Sibener LV, et al. (2018) Isolation of a structural [83] Cendrowski J, Maminska A, Miaczynska M (2016) En- mechanism for uncoupling t cell receptor signaling from docytic regulation of cytokine receptor signaling. Cy- peptide-mhc binding. Cell 174:672–687 e27. tokine Growth Factor Rev 32:63–73. [64] Wu P, et al. (2019) Mechano-regulation of peptide-mhc [84] Tkach KE, et al. (2014) T cells translate individual, class i conformations determines tcr antigen recognition. quantal activation into collective, analog cytokine re- Mol Cell 73:1015–1027 e7. sponses via time-integrated feedbacks. Elife 3:e01944. 73

[85] Voisinne G, et al. (2015) T cells integrate local and fates. Cell 126:755–66. global cues to discriminate between structurally similar [105] Kueh HY, Champhekar A, Nutt SL, Elowitz MB, antigens. Cell Rep 11:1208–19. Rothenberg EV (2013) Positive feedback between pu.1 [86] Polonsky M, et al. (2018) Induction of CD4 T cell mem- and the cell cycle controls myeloid differentiation. Sci- ory by local cellular collectivity. Science 360. ence 341:670–3. [87] Becker V, et al. (2010) Covering a broad dynamic range: [106] Walczak AM, Sasai M, Wolynes PG (2005) Self- information processing at the erythropoietin receptor. consistent proteomic field theory of stochastic gene Science 328:1404–8. switches. Biophysical journal 88:828–50. [88] Raue A, Kreutz C, Maiwald T, Klingmuller U, Timmer [107] Walczak AM, Onuchic JN, Wolynes PG (2005) Absolute J (2011) Addressing parameter identifiability by model- rate theories of epigenetic stability. Proceedings of the based experimentation. IET Syst Biol 5:120–30. National Academy of Sciences of the United States of [89] Bachmann J, et al. (2011) Division of labor by dual America 102:18926–31. feedback regulators controls jak2/stat5 signaling over [108] Friedman N, Cai L, Xie XS (2006) Linking stochas- broad ligand range. Mol Syst Biol 7:516. tic dynamics to population distribution: an analytical [90] Karr JR, et al. (2015) Summary of the dream8 pa- framework of gene expression. Phys Rev Lett 97:168302. rameter estimation challenge: Toward parameter iden- [109] Elowitz MB, Levine AJ, Siggia ED, Swain PS (2002) tification for whole-cell models. PLoS Comput Biol Stochastic gene expression in a single cell. Science 11:e1004096. 297:1183–6. [91] Shi T, et al. (2016) Conservation of protein abundance [110] Feinerman O, Veiga J, Dorfman JR, Germain RN, patterns reveals the regulatory architecture of the egfr- Altan-Bonnet G (2008) Variability and robustness in mapk pathway. Sci Signal 9:rs6. t cell activation from regulated heterogeneity in protein [92] Mitchell S, Roy K, Zangle TA, Hoffmann A (2018) Non- levels. Science 321:1081–4. genetic origins of cell-to-cell variability in b lymphocyte [111] Szabo SJ, et al. (2000) A novel transcription factor, t- proliferation. Proc Natl Acad Sci U S A 115:E2888– bet, directs th1 lineage commitment. Cell 100:655–69. E2897. [112] Palma A, Jarrah AS, Tieri P, Cesareni G, Cas- [93] Thurley K, Gerecht D, Friedmann E, Hofer T (2015) tiglione F (2018) Gene regulatory network modeling of Three-dimensional gradients of cytokine signaling be- macrophage differentiation corroborates the continuum tween t cells. PLoS Comput Biol 11:e1004206. hypothesis of polarization states. Front Physiol 9:1659. [94] Berezhkovskii AM, Sample C, Shvartsman SY (2010) [113] Kueh HY, et al. (2016) Asynchronous combinatorial How long does it take to establish a morphogen gradi- action of four regulatory factors activates bcl11b for t ent? Biophys J 99:L59–61. cell commitment. Nat Immunol 17:956–65. [95] Kolomeisky A (2011) Formation of a morphogen gradi- [114] Peine M, et al. (2013) Stable t-bet(+)gata-3(+) th1/th2 ent: Acceleration by degradation. J. Phys. Chem. Lett. hybrid cells arise in vivo, can develop directly from naive 2:1502–1505. precursors, and limit immunopathologic inflammation. [96] Oyler-Yaniv A, et al. (2017) A Tunable Diffusion- PLoS Biol 11:e1001633. Consumption Mechanism of Cytokine Propagation En- [115] Moran U, Phillips R, Milo R (2010) SnapShot: key ables Plasticity in Cell-to-Cell Communication in the numbers in biology. Cell 141:1262–1262.e1. Immune System. Immunity 46:609–620. [116] Milo R, Phillips R (2015) Cell Biology by the Numbers [97] Marcou Q, et al. (2018) A model for the integration of (Garland Science), p 2015. conflicting exogenous and endogenous signals by den- [117] Busch K, et al. (2015) Fundamental properties of unper- dritic cells. Phys. Biol. 15. turbed haematopoiesis from stem cells in vivo. Nature [98] Huang J, et al. (2013) A single peptide-major histo- 518:542. compatibility complex ligand triggers digital cytokine [118] HöferT, Barile M, Flossdorf M (2016) Stem-cell dynam- secretion in cd4(+) t cells. Immunity 39:846–57. ics and lineage topology from in vivo fate mapping in [99] Zhu J, Yamane H, Paul WE (2010) Differentiation of the hematopoietic system. Current opinion in biotech- effector cd4 t cell populations (*). Annu Rev Immunol nology 39:150–156. 28:445–89. [119] Zilman A, Ganusov VV, Perelson AS (2010) Stochastic [100] Chaouat G, et al. (2004) Th1/th2 paradigm in preg- models of lymphocyte proliferation and death. PLoS nancy: paradigm lost? cytokines in pregnancy/early One 5:1–14. abortion: reexamining the th1/th2 paradigm. Int Arch [120] Perelson AS (2002) Modelling Viral and Immune Sys- Allergy Immunol 134:93–119. tem Dynamics. Nat. Rev. Immunol. 2:28–36. [101] Yates A, Callard R, Stark J (2004) Combining cytokine [121] Yates A, et al. (2007) Reconstruction of cell population signalling with t-bet and gata-3 regulation in th1 and dynamics using CFSE. BMC Bioinformatics 20:1–20. th2 differentiation: a model for cellular decision-making. [122] Schoedel KB, et al. (2016) The bulk of the hematopoi- J Theor Biol 231:181–96. etic stem cell population is dispensable for murine [102] Antebi YE, et al. (2013) Mapping differentiation under steady-state and stress hematopoiesis. Blood 128:2285– mixed culture conditions reveals a tunable continuum 2297. of t cell fates. PLoS Biol 11:e1001616. [123] Perie L, et al. (2014) Determining Lineage Pathways [103] Hong T, Xing J, Li L, Tyson JJ (2011) A mathematical from Cellular Barcoding Experiments. Cell Reports model for the reciprocal differentiation of t helper 17 6:617–624. cells and induced regulatory t cells. PLoS Comput Biol [124] Ogawa M, Porter PN, Nakahata T (1983) Renewal And 7:e1002122. Commitment To Differentiation Of Hemopoietic Stem [104] Laslo P, et al. (2006) Multilineage transcriptional prim- Cells ( An Interpretive Review ). Blood 5:823–829. ing and determination of alternate hematopoietic cell 74

[125] Tsuji K, Nakahata T (1989) Stochastic Model for Multi- sight into the human immunoglobulin repertoire. Proc. potent Hemopoietic Progenitor Differentiation. Journal Natl. Acad. Sci. 106:20216–20221. of cellular physiology 653:647–653. [144] Han A, Glanville J, Hansmann L, Davis MM (2014) [126] Pei W, et al. (2017) Polylox barcoding reveals Linking T-cell receptor sequence to functional pheno- haematopoietic stem cell fates realized in vivo. Nature type at the single-cell level. Nat. Biotechnol. 32:684– Publishing Group 548:456–460. 692. [127] Laffleur B, et al. (2014) Immunoglobulin genes undergo [145] Dupic T, Marcou Q, Walczak AM, Mora T (2018) Gen- legitimate repair in human B cells not only after cis - but esis of the αβ T-cell receptor. arXiv:1806.11030. also frequent trans -class switch recombination. Genes [146] Arstila TP, et al. (1999) A direct estimate of the human and Immunity pp 341–346. alphabeta T cell receptor diversity. Science 286:958– [128] Murphy K, Travers P, Walport M (2007) Janeway’s 961. Immunology (Garland Science), 7 edition edition. [147] Weinstein JA, Jiang N, White RA, Fisher DS, Quake [129] Murphy KM, et al. (2000) Signaling AND Transcription SR (2009) High-throughput sequencing of the zebrafish in T Helper development. Annu. Rev. Immunol. 18:451– antibody repertoire. Science (80-. ). 324:807–810. 494. [148] Robins HS, et al. (2009) Comprehensive assessment of [130] Hawkins ED, Turner ML, Dowling MR, Gend CV, T-cell receptor beta-chain diversity in alphabeta T cells. Hodgkin PD (2007) A model of immune regulation as Blood 114:4099–4107. a consequence of randomized lymphocyte division and [149] Boyd SD, et al. (2009) Measurement and clinical moni- death times. Proceedings of the National Academy of toring of human lymphocyte clonality by massively par- Sciences 104:5032–5037. allel {VDJ} pyrosequencing. Sci Transl Med 1:12ra23. [131] Marchingo JM, et al. (2014) Antigen affinity, costim- [150] Benichou J, Ben-Hamo R, Louzoun Y, Efroni S (2012) ulation, and cytokine inputs sum linearly to. Science Rep-Seq: Uncovering the immunological repertoire 346:1123–1128. through next-generation sequencing. [132] Marchingo JM, et al. (2016) T-cell stimuli indepen- [151] Six A, et al. (2013) The past, present and future of dently sum to regulate an inherited clonal division fate. immune repertoire biology - the rise of next-generation Nature Communications 7:1–12. repertoire analysis. Front. Immunol. 4:413. [133] Duffy KR, Hodgkin PD (2012) Intracellular competition [152] Robins H (2013) Immunosequencing: applications of for fates in the immune system. Trends in Cell Biology immune repertoire deep sequencing. Curr. Opin. Im- 22:457–464. munol. 25:646–652. [134] Deenick EK, Gett AV, Hodgkin PD (2003) Stochastic [153] Georgiou G, et al. (2014) The promise and challenge of Model of T Cell Proliferation: A Calculus Revealing IL- high-throughput sequencing of the antibody repertoire. 2 Regulation of Precursor Frequencies, Cell Cycle Time, Nat. Biotechnol. 32:158–68. and Survival. The Journal of Immunology 170:4963– [154] Heather JM, Ismail M, Oakes T, Chain B (2017) High- 4972. throughput sequencing of the T-cell receptor repertoire: [135] Tangye SG, Avery DT, Deenick EK, Hodgkin PD (2003) pitfalls and opportunities. Brief. Bioinform. p bbw138. Intrinsic Differences in the Proliferation of Naive and [155] Minervina A, Pogorelyy M, Mamedov I (2019) TCR and Memory Human B Cells as a Mechanism for Enhanced BCR repertoire profiling in adaptive immunity. Transpl. Secondary Immune Responses. The Journal of Im- Int. pp 0–2. munology 170:686–694. [156] Rubelt F, et al. (2017) Adaptive Immune Receptor [136] Bendall SC, Nolan GP, Roederer M, Chattopadhyay PK Repertoire Community recommendations for sharing (2012) A deep profiler ’ s guide to cytometry. Trends in immune-repertoire sequencing data. Nature immunol- Immunology 33:323–332. ogy 18:1274–1278. [137] Buchholz VR, et al. (2013) Disparate Individual Fates [157] Fisher R, Steven Corbet A, Williams C (2016) The Re- Compose Robust CD8+ T Cell Immunity. Science lation Between the Number of Species and the Number 340:630–636. of Individuals in a Random Sample of an Animal Pop- [138] Flossdorf M, RösslerJ, Buchholz VR, Busch DH, Höfer ulation. J. Anim. Ecol. 12:42–58. T (2015) CD8 + T cell diversification by asymmetric [158] Qi Q, et al. (2014) Diversity and clonal selection in the cell division. Nature immunology 16:891–893. human T-cell repertoire. Proc. Natl. Acad. Sci. U. S. [139] Dowling MR, et al. (2014) Stretched cell cycle model A. 111:13139–44. for proliferating lymphocytes. Proc Natl Acad Sci U S [159] Chao A, Bunge J (2002) Estimating the number of A 111:6377–6382. species in a stochastic abundance model. Biometrics [140] Miles AS, Hodgkin PD, Duffy KR (2018) Inferring dif- 58:531–539. ferentiation order in adaptive immune responses from [160] DeWitt WS, et al. (2016) A Public Database of Mem- population level data. Springer Volume on T-cell im- ory and Naive B-Cell Receptor Sequences. PLoS One munology. 11:e0160853. [141] Kinjyo I, et al. (2015) Real-time tracking of cell cycle [161] Lythe G, Callard RE, Hoare R, Molina-Par´ısC (2015) progression during CD8+ effector and memory T-cell How many TCR clonotypes does a body maintain? J. differentiation. Nature Communications 6:1–13. Theor. Biol. 389:214–224. [142] Jenkins MK, Chu HH, McLachlan JB, Moon JJ (2009) [162] Mora T, Walczak A (2018) in Syst. Immunol., eds Das On the composition of the preimmune repertoire of T JD, Jayaprakash C (CRC Press), pp 185–199. cells specific for Peptide-major histocompatibility com- [163] Wardemann H, et al. (2003) Predominant Autoantibody plex ligands. Annu. Rev. Immunol. 28:275–294. Production by Early Human B Cell Precursors. Science [143] Glanville J, et al. (2009) Precise determination of the (80-. ). 301:1374–1377. diversity of a combinatorial antibody library gives in- 75

[164] Lauemøller SL, et al. (2003) Sensitive quantitative pre- Elife 7:1–13. dictions of peptide-MHC binding by a Query by Com- [183] Pogorelyy MV, et al. (2018) Detecting T-cell recep- mittee ’ artificial neural network approach. Tissue Anti- tors involved in immune responses from single repertoire gens 62:378–384. snapshots. arXiv:1807.08833. [165] Jensen KK, et al. (2018) Improved methods for predict- [184] Wing K, Sakaguchi S (2010) Regulatory T cells exert ing peptide binding affinity to MHC class II molecules. checks and balances on self tolerance and autoimmunity. Immunology 154:394–406. Nat. Immunol. 11:7–13. [166] Moon JJ, et al. (2007) Naive CD4+ T Cell Frequency [185] Bains I, van Santen HM, Seddon B, Yates AJ (2013) Varies for Different Epitopes and Predicts Repertoire Models of Self-Peptide Sampling by Developing T Cells Diversity and Response Magnitude. Immunity 27:203– Identify Candidate Mechanisms of Thymic Selection. 213. PLoS Comput. Biol. 9. [167] Jenkins MK, Moon JJ (2012) The role of naive {T} [186] Le Borgne M, et al. (2009) The impact of negative cell precursor frequency and recruitment in dictating selection on thymocyte migration in the medulla. Nat. immune response magnitude. J. Immunol. 188:4135– Immunol. 10:823–830. 4140. [187] Kosmrlj A, et al. (2010) Effects of thymic selection of [168] Yates AJ (2014) Theories and quantification of thymic the T-cell repertoire on HLA class I-associated control selection. Front. Immunol. 5:13. of HIV infection. Nature 465:350–354. [169] Murugan A, Mora T, Walczak AM, Callan CG (2012) [188] Elhanati Y, Murugan A, Callan CG, Mora T, Walczak Statistical inference of the generation probability of T- AM (2014) Quantifying selection in immune receptor cell receptors from sequence repertoires. Proc. Natl. repertoires. Proc. Natl. Acad. Sci. 111:9875–9880. Acad. Sci. 109:16161–16166. [189] Kaplinsky J, et al. (2014) Antibody repertoire deep se- [170] Elhanati Y, et al. (2015) Inferring processes underlying quencing reveals antigen-independent selection in ma- B-cell repertoire diversity. Philos Trans R Soc Lond, B, turing B cells. Proc. Natl. Acad. Sci. 111:E2622–9. Biol Sci 370:20140243. [190] Elhanati Y, Sethna Z, Callan CG, Mora T, Walczak [171] Pogorelyy MV, et al. (2017) Method for identification of AM (2018) Predicting the spectrum of TCR repertoire condition-associated public antigen receptor sequences. sharing with a data-driven model of recombination. Im- arXiv:1709.09703. munol Rev 284:167–179. [172] Toledano A, et al. (2018) Evidence for shaping of light [191] Mora T, Walczak AM (2016) Renyi entropy, abundance chain repertoire by structural selection. Front. Im- distribution and the equivalence of ensembles. Phys Rev munol. 9:1–9. E 95:052418. [173] Sethna Z, et al. (2017) Insights into immune system de- [192] Mora T, Walczak AM, Bialek W, Callan CG (2010) velopment and function from mouse T-cell repertoires. Maximum entropy models for antibody diversity. Proc. Proc. Natl. Acad. Sci. 114:2253–2258. Natl. Acad. Sci. 107:5405–5410. [174] Magadan S, et al. (2018) Origin of Public Memory B [193] Desponds J, Mora T, Aleksandra W (2016) Fluctuat- Cell Clones in Fish After Antiviral Vaccination. Front. ing fitness shapes the clone size distribution of immune Immunol. 9:2115. repertoires. Proc Natl Acad Sci USA 113:274. [175] Marcou Q, Mora T, Walczak AM (2018) High- [194] Desponds J, Mayer A, Mora T, Walczak AM (2017) throughput immune repertoire analysis with IGoR. Nat. Population dynamics of immune repertoires. pp 1–9. Commun. 9:561. [195] Greef PCD, et al. (2019) The naive T-cell receptor [176] Sethna Z, Elhanati Y, Callan CG, Walczak AM, Mora repertoire has an extremely broad distribution of clone T (2019) OLGA: fast computation of generation prob- sizes. bioRxiv. abilities of B- and T-cell receptor amino acid sequences [196] Best K, Oakes T, Heather JM, Shawe-Taylor J, Chain and motifs. Bioinformatics btz035. B (2015) Computational analysis of stochastic hetero- [177] Munshaw S, Kepler TB (2010) SoDA2: a Hid- geneity in PCR amplification efficiency revealed by sin- den Markov Model approach for identification of im- gle molecule barcoding. Sci. Rep. 5:14629. munoglobulin rearrangements. Bioinformatics 26:867– [197] Vollmers C, Sit RV, Weinstein Ja, Dekker CL, Quake 72. SR (2013) Genetic measurement of memory B-cell recall [178] Elhanati Y, Marcou Q, Mora T, Walczak AM (2015) using antibody repertoire sequencing. Proc. Natl. Acad. repgenHMM: a dynamic programming tool to infer the Sci. 110:13463–8. rules of immune receptor generation from sequence data. [198] Shugay M, et al. (2014) Towards error-free profiling of bioRxiv In press:1–9. immune repertoires. Nat. Methods 11:653–5. [179] Ralph DK, Matsen FA (2016) Consistency of VDJ Rear- [199] Kaplinsky J, Arnaout R (2016) Robust estimates rangement and Substitution Parameters Enables Accu- of overall immune-repertoire diversity from high- rate B Cell Receptor Sequence Annotation. PLoS Com- throughput measurements on samples. Nat. Commun. put. Biol. 12:1–25. 7:11881. [180] Ralph DK, Matsen FA (2016) Likelihood-Based Infer- [200] Laydon DJ, Bangham CRM, Asquith B, Crm B (2015) ence of B Cell Clonal Families. PLoS Comput. Biol. Estimating T-cell repertoire diversity: limitations of 12:1–28. classical estimators and a new approach. Philos Trans [181] Wang Y, et al. (2011) Genomic screening by 454 py- R Soc Lond, B, Biol Sci 370:20140291. rosequencing identifies a new human IGHV gene and [201] Haegeman B, et al. (2013) Robust estimation of mi- sixteen other new IGHV allelic variants. Immunogenet- crobial diversity in theory and in practice. ISME J. ics 63:259–265. 7:1092–1101. [182] Pogorelyy MV, et al. (2018) Method for identification of [202] Emerson RO, et al. (2017) Immunosequencing identi- condition-associated public antigen receptor sequences. fies signatures of cytomegalovirus exposure history and 76

HLA-mediated effects on the T cell repertoire. Nat. [223] Zheng GX, et al. (2017) Massively parallel digital tran- Genet. 49:659–665. scriptional profiling of single cells. Nat. Commun. 8:1– [203] Mayer A, Balasubramanian V, Mora T, Walczak AM 12. (2015) How a well-adapted immune system is organized. [224] Hubbell SP (2001) The Unified Neutral Theory of Bio- Proceedings of the National Academy of Sciences USA diversity and Biogeography. pp 1–15. [225] Jerne N (1974) Towards a network theory of the immune [204] Emerson RO, et al. (2017) Immunosequencing identi- system. Ann Immunol (Paris) 125C:373–89. fies signatures of cytomegalovirus exposure history and [226] De Boer RJ, Perelson AS (1994) T cell repertoires and HLA-mediated effects on the T cell repertoire. Nat. competitive exclusion. Genet. 49:659–665. [227] De Boer RJ, Perelson AS (1997) Competitive control [205] Faham M, et al. (2017) Discovery of T Cell Re- of the self-renewing T cell repertoire. Int. Immunol. ceptor β Motifs Specific to HLAB27Positive Ankylos- 9:779–790. ing Spondylitis by Deep Repertoire Sequence Analysis. [228] De Boer RJ, Freitas aa, Perelson aS (2001) Resource Arthritis Rheumatol. 69:774–784. competition determines selection of B cell repertoires. [206] Venturi V, et al. (2006) Sharing of T cell receptors J. Theor. Biol. 212:333–343. in antigen-specific responses is driven by convergent re- [229] Mayer A, Balasubramanian V, Walczak AM, Mora T combination. Proc. Natl. Acad. Sci. 103:18691–18696. (2018) How a well-adapting immune system remembers. [207] Venturi V, et al. (2011) A mechanism for TCR shar- arXiv:1806.05753v2. ing between T cell subsets and individuals revealed by [230] Perelson AS, Ribeiro RM (2013) Modeling the within- pyrosequencing. J. Immunol. 186:4285–4294. host dynamics of HIV infection. BMC Biology 11:96. [208] Madi A, et al. (2014) T-cell receptor repertoires share a [231] Perelson AS, et al. (1996) HIV-1 Dynamics in Vivo : restricted set of public and abundant CDR3 sequences Virion Clearance Rate , Infected Cell Life-Span , and that are associated with self-related immunity. Genome Viral Generation Time. Science 271:1582–1586. Res. 24:1603–12. [232] Perelson AS, et al. (1997) Decay characteristics of HIV- [209] Perelson A, Weisbuch G (1997) Immunology for physi- 1-infected compartments during combination therapy. cists. Reviews of Modern Physics 69:1219–1268. Nature 38718:188–191. [210] Mayer A, Balasubramanian V, Mora T, Walczak AM [233] Mccoy CO, et al. (2015) Quantifying evolutionary con- (2015) How a well-adapted immune system is organized. straints on B-cell affinity maturation. Philos Trans R Proc. Natl. Acad. Sci. 112:5950–5955. Soc Lond, B, Biol Sci 370:20140244. [211] Thomas N, et al. (2014) Tracking global changes in- [234] Yaari G, Kleinstein SH (2015) Practical guidelines for duced in the CD4 T-cell receptor repertoire by immu- B-cell receptor repertoire sequencing analysis. Genome nization with a complex antigen using short stretches of Med. 7:121. CDR3 protein sequence. Bioinformatics 30:3181–3188. [235] Hoehn KB, Fowler A, Lunter G, Pybus OG (2016) [212] Cinelli M, et al. (2017) Feature selection using a one The Diversity and Molecular Evolution of B-Cell Recep- dimensional na¨ıve Bayes’ classifier increases the accu- tors during Infection. Molecular Biology and Evolution racy of support vector machine classification of CDR3 33:1147–1157. repertoires. Bioinformatics 33:btw771. [236] Gupta NT, et al. (2015) Change-O: A toolkit for ana- [213] Castro R, et al. (2013) Teleost Fish Mount Complex lyzing large-scale B cell immunoglobulin repertoire se- Clonal IgM and IgT Responses in Spleen upon Systemic quencing data. Bioinformatics 31:3356–3358. Viral Infection. PLoS Pathog. 9. [237] Vander Heiden JA, et al. (2014) PRESTO: A toolkit [214] Laserson U, et al. (2014) High-resolution antibody dy- for processing high-throughput sequencing raw reads namics of vaccine-induced immune responses. Proc. of lymphocyte receptor repertoires. Bioinformatics Natl. Acad. Sci. 111:4928–4933. 30:1930–1932. [215] Wang C, et al. (2015) B-cell repertoire responses to [238] Yaari G, Benichou JIC, Heiden JAV, Kleinstein SH, varicella-zoster vaccination in human identical twins. Louzoun Y (2015) The mutation patterns in B-cell im- Proc. Natl. Acad. Sci. 112:500–505. munoglobulin receptors reflect the influence of selection [216] Qi Q, et al. (2016) Diversification of the antigen-specific acting at multiple time-scales. Philos Trans R Soc Lond, T cell receptor repertoire after varicella zoster vaccina- B, Biol Sci 370:20140242. tion. Sci. Transl. Med. 8:332ra46–332ra46. [239] Cui A, et al. (2016) A Model of Somatic Hypermuta- [217] DeWitt WS, et al. (2015) Dynamics of the Cytotoxic T tion Targeting in Mice Based on High-Throughput Ig Cell Response to a Model of Acute Viral Infection. J. Sequencing Data. Journal of immunological methods. Virol. 249:JVI.03474–14. [240] Yaari G, et al. (2013) Models of somatic hypermutation [218] Pogorelyy MV, et al. (2018) Precise tracking of vaccine- targeting and substitution based on synonymous muta- responding T-cell clones reveals convergent and person- tions from high-throughput immunoglobulin sequencing alized response in identical twins. arXiv:1804.04485. data. Frontiers in immunology 4:1–11. [219] Borghans JAM, De Boer RJ (2007) Quantification of [241] Dhar A, Davidsen K, Matsen FA, Minin VN (2018) T-cell dynamics: From telomeres to DNA labeling. Im- Predicting B Cell Receptor Substitution Profiles Using munol. Rev. 216:35–47. Public Repertoire Data. PLoS Genet. [220] De Boer RJ, Perelson AS (2013) Quantifying T lym- [242] DeWitt WS, Mesin L, Victora GD, Minin VN, Matsen phocyte turnover. J. Theor. Biol. 327:45–87. FA (2018) Using genotype abundance to improve phy- [221] Bains I, Antia R, Callard R, Yates AJ (2009) Quan- logenetic inference. Mol. Biol. Evol. 35:1253–1265. tifying the development of the peripheral naive CD4+ [243] Jacob J, Kassir R, Kelsoe G (1991) In Situ Studies T-cell pool in humans. Blood 113:5480–7. of the Primary Immune Response to ( 4-hydroxy-3- [222] Murugan A (2012) private communication. nitrophenyl ) acetyl . I . The Architecture and Dynamics 77

of Responding Cell Populations. Journal of Experimen- [263] Dunn-walters DK, Hare JSO (2018) in Handbook of tal Medicine 173. Immunosenescence, ed T. Fulop et al. (eds.) (Springer [244] Nieuwenhuis P, Opstelten D (1984) Functional International Publishing AG 2018). Anatomy of Germinal Centers. American Journal of [264] Wendel BS, et al. (2017) Accurate immune repertoire Anatomy 435:421–435. sequencing reveals malaria infection driven antibody lin- [245] Shapiro GS, Aviszus K, Ikle D, Wysocki LJ (1999) Pre- eage diversi fi cation in young children. Nature Com- dicting regional mutability in antibody v genes based munications 8:531. solely on di-and trinucleotide sequence composition. [265] Nourmohammad A, Otwinowski J, Luksza M, Mora T, The Journal of Immunology 163:259–268. Walczak AM (2018) Clonal competition in B-cell reper- [246] Uduman M, et al. (2011) Detecting selection in im- toires during chronic HIV-1 infection. bioarxiv. munoglobulin sequences. Nucleic Acids Res. 39:W499– [266] Horns F, Vollmers C, Dekker CL, Quake SR (2019) W504. Signatures of selection in the human antibody repertoire [247] Yaari G, et al. (2013) Models of somatic hypermutation : Selective sweeps , competing subclones , and neutral targeting and substitution based on synonymous muta- drift. Proceedings of the National Academy of Sciences tions from high-throughput immunoglobulin sequencing 116. data. Front. Immunol. 4:358. [267] Vieira MC, Zinder D, Cobey S (2018) Selection and [248] Feng J, Shaw DA, Minin VN, Simon N, Matsen IV FA Neutral Mutations Drive Pervasive Mutability Losses (2017) Survival analysis of DNA mutation motifs with in Long-Lived Anti-HIV B-Cell Lineages. Molecular Bi- penalized proportional hazards. arxiv 1711.04057. ology and Evolution 35:1135–1146. [249] Unniraman S, Schatz DG (2007) Strand-Biased Spread- [268] Nourmohammad A, Otwinowski J, Plotkin JB (2016) ing of Mutations During Somatic Hypermutation. Sci- Host-Pathogen Coevolution and the Emergence of ence (80-. ). 317:1227–1230. Broadly Neutralizing Antibodies in Chronic Infections. [250] Oprea M, Perelson AS (1997) Somatic Mutation Leads PLoS genetics 12:e1006171. to E cient A nity Maturation When Centrocytes Recy- [269] Blanquart F, Gandon S (2013) Time-shift experiments cle Back to Centroblasts. Journal of immunology (Bal- and patterns of adaptation across time and space. Ecol- timore, Md. : 1950) 158:5155–62. ogy Letters pp 31–38. [251] Oprea M, van Nimwegen E, Perelson AS (2000) Dynam- [270] Richman DD, Wrin T, Little SJ, Petropoulos CJ (2003) ics of One-pass Germinal Center Models : Implications Rapid evolution of the neutralizing antibody response for Affinity Maturation. Bulletin of Mathematical Biol- to HIV type 1 infection. Proceedings of the National ogy pp 121–153. Academy of Sciences 100:4144–4149. [252] Victora GD, Nussenzweig MC (2012) Germinal centers. [271] Frost SDW, et al. (2005) Neutralizing antibody re- Annu. Rev. Immunol. 30:429–457. sponses drive the evolution of human immunodefi- [253] Berek C, Milstein C (1987) Mutation drift and reper- ciency virus type 1 envelope during recent HIV infec- toire shift in the maturation of the immune response. tion. Proceedings of the National Academy of Sciences Immunol. Rev. 96:23–41. 102:18514–18519. [254] Eisen HN, Siskind GW (1964) Variations in Affinities of [272] Moore PL, et al. (2009) Limited Neutralizing Antibody Antibodies during the Immune Response. Biochemistry Specificities Drive Neutralization Escape in Early HIV-1 3:996–1008. Subtype C Infection. PLoS Pathogens 5. [255] Tas JMJ, et al. (2016) Visualizing antibody affinity [273] Luksza M, LässigM (2014) A predictive fitness model maturation in germinal centers. Science 3439:1–11. for influenza. Nature 507:57–61. [256] Kuraoka M, et al. (2016) Complex Antigens Drive Per- [274] Ferguson AL, et al. (2013) Translating HIV Sequences missive Clonal Selection in Germinal Centers. Immunity into Quantitative Fitness Landscapes Predicts Viral 44:542–552. Vulnerabilities for Rational Immunogen Design. Immu- [257] Victora GD, Nussenzweig MC (2012) Germinal Centers. nity 38:606–617. Annu. Rev. Immunol. 2012. 30:429–57. [275] Shekhar K, et al. (2013) Spin models inferred from [258] Victora GD, et al. (2010) Germinal center dynamics patient-derived viral sequence data faithfully describe revealed by multiphoton microscopy with a photoacti- HIV fitness landscapes. Physical Review E 88:62705. vatable fluorescent reporter. Cell 143:592–605. [276] Barton JP, et al. (2016) Relative rate and location of [259] Lee J, et al. (2016) Molecular-level analysis of the serum intra-host HIV evolution to evade cellular immunity are antibody repertoire in young adults before and after sea- predictable. Nature Communications 7:1–10. sonal influenza vaccination. Nature medicine. [277] Eigen M (1971) Selforganization of Matter and the [260] Wang S, Burton D, Kardar M, Chakraborty A (2014) Evolution of Biological Macromolecules. Die Naturwis- Guiding the evolution to catch the virus: An in silico senschaften. study of affinity maturation against rapidly mutating [278] LeuthäusserI, Leuthausser I (1998) An exact corre- antigen. Bulletin of the American Physical Society Vol- spondence between Eigen ’ s evolution model and a two- ume 59, Number 1. dimensional Ising system. Journal of Chemical Physics [261] Murugan R, et al. (2018) Clonal selection drives pro- 1884:1985–1987. tective memory B cell responses in controlled human [279] Phillips R, Kondev J, Theriot J, Garcia H (2012) Phys- malaria infection. Science Immunology pp 1–10. ical Biology of the Cell p 2012. [262] Neu KE, et al. (2019) Spec-seq unveils transcriptional [280] Zanini F, Brodin J, Thebo L, Lanz C, Albert J (2015) subpopulations of antibody-secreting cells following in- Population genomics of intrapatient HIV-1 evolution. fluenza vaccination. The Journal of Clinical Investiga- eLife pp 1–26. tionJournal of Clinical Investigation 129:93–105. [281] Smith DJ, Lapedes AS, Jong JCD (2004) Mapping the antigenic and genetic evolution of influenza virus. Sci- 78

ence 305:371–377. [304] Everitt B (2011) Cluster analysis (Wiley, Chichester, [282] Neher RA, Russell CA, Shraiman BI (2014) Predicting West Sussex, U.K). evolution from the shape of genealogical trees. eLife [305] Yang Z (2006) Computational Molecular Evolution (Ox- 3:1–18. ford University Press). [283] Bao Y, et al. (2008) The influenza virus resource at the [306] Stamatakis A (2014) RAxML version 8 : a tool for phy- National Center for Biotechnology Information. Journal logenetic analysis and post-analysis of large phylogenies. of virology 82:596–601. Bioinformatics (Oxford, England) 30:1312–1313. [284] Grenfell BT (2008) Measles: Nonlinearity and Stochas- [307] Hoehn KB, Lunter G, Pybus OG (2017) A Phylogenetic ticity in an Epidemic Metapopulation (Oxford Univer- Codon Substitution Model for Antibody Lineages. Ge- sity Press). netics 206:417–427. [285] Rouzine IM, Rozhnova G (2018) Antigenic evolution of [308] Davidsen K, Matsen IV FA (2018) Benchmarking Tree viruses in host populations. PLoS Pathogens pp 1–16. and Ancestral Sequence Inference for B Cell Receptor [286] Yan L, Neher R, Shraiman BI (2018) Phylodynamics of Sequences. Frontiers in immunology 9:1–13. rapidly adapting pathogens: extinction and speciation [309] Kussell E, Leibler S (2005) Phenotypic diversity, pop- of a Red Queen. pp 1–15. ulation growth, and information in fluctuating environ- [287] Desai MM, Fisher DS (2007) Beneficial mutation selec- ments. Science 309:2075–8. tion balance and the effect of linkage on positive selec- [310] Rivoire O, Leibler S (2011) The Value of Information tion. Genetics 176:1759–98. for Populations in Varying Environments. Journal of [288] Bedford T, Rambaut A, Pascual M (2012) Canalization Statistical Physics 142:1124–1166. of the evolutionary trajectory of the human influenza [311] Bradde S, Mora T, Walczak AM (2019) Cost and bene- virus. BMC Biology 10:38. fits of CRISPR spacer acquisition. Philosophical Trans- [289] Marchi J, LässigM, Mora T, Walczak AM (2019) Multi- actions of the Royal Society B: Biological Sciences pp lineage evolution in viral populations driven by host im- 1–10. mune systems. arXiv:1906.07444. [312] Lotka AJ (1920) Analytical note on certain rhythmic [290] Regev A, et al. (2017) The Human Cell Atlas. eLife relations in organic systems. Proc. Nat. Acad 6:410–415. 6:e27041. [313] Volterra V (1956) Opere mathematiche. Memorie e [291] Mehta P, et al. (2018) A high-bias, low-variance intro- Note. ed dei Lincei AN (Accademia Nazionale dei Lin- duction to Machine Learning for physicists. cei, Rome), pp xxxiii + 604. [292] Walczak A, Mugler A, Wiggins C (2012) in Computa- [314] Edelstein-Keshet L (2005) Mathematical Models in Bi- tional Modeling of Signaling Networks pp 273–322. ology (SIAM Philadelphia). [293] TkaˇcikG, Walczak AM (2011) Information transmis- [315] May RM (2001) Stability and complexity in model sion in genetic regulatory networks: a review. Journal ecosystems (Princeton University Press). of physics. Condensed matter : an Institute of Physics [316] Marchenko V, Pastur L (1967) Distribution of eigenval- journal 23:153102. ues for some sets of random matrices. Mat. Sb. 72:507– [294] for Immunology TJS, Ishikawa T (2009) Your Amazing 536. Immune System. [317] Yule UG (1925) A mathematical theory of evolution, [295] Hartl DL (2011) Genetics: Analysis of Genes and based on the conclusions of Dr J.C. Willis, F.R.S. Phil Genomes ed Learning J&B. Trans R Soc Lond, Ser B CCXIII. [296] Gillespie, John H (2004) Population Genetics : A Con- [318] Simon HA (1955) On a Class of Skew Distribution Func- cise Guide (Johns Hopkins University Press), p 2004. tions. Biometrika Trust 42:425–440. [297] Takahata N, Ishii K, Matsuda H (1975) Effect of tempo- [319] Kepler TB, et al. (2014) Reconstructing a B-cell Clonal ral fluctuation of selection coefficient on gene frequency Lineage. II. Mutation, Selection, and Affinity Matura- in a population. Proc. Natl. Acad. Sci. U.S.A. 72:4541– tion. Front. Immunol. 5. 4545. [320] Elhanati Y, Marcou Q, Mora T, Walczak AM (2016) [298] Nourmohammad A, Schiffels S, LässigM (2013) Evolu- RepgenHMM: A dynamic programming tool to infer the tion of molecular phenotypes under stabilizing selection. rules of immune receptor generation from sequence data. J. Stat. Mech. Theory Exp. 2013. Bioinformatics 32:1943–1951. [299] Allen LJS (2003) An Introduction to Stochastic Pro- [321] PresséS, et al. (2013) Principles of maximum entropy cesses With Biology Applications . (Prentice-Hall, New and maximum caliber in statistical physics. Rev. Mod. York). Phys. 85:1115–1141. [300] Zapperi S, Ba K, Stanley HE (1995) Self- [322] Barton JP, De Leonardis E, Coucke A, Cocco S (2016) Organized Branching Processes: Mean-Field Theory for ACE: Adaptive cluster expansion for maximum entropy Avalanche. Phys. Rev. Lett. 75:4071–4074. graphical model inference. Bioinformatics 32:3089– [301] Beggs JM, Plenz D (2003) Neuronal avalanches in neo- 3097. cortical circuits. J. Neurosci. 23:11167–11177. [323] Nguyen HC, Zecchina R, Berg J (2017) Inverse statis- [302] Neher RA (2013) Genetic Draft, Selective Interference, tical problems: from the inverse Ising problem to data and Population Genetics of Rapid Adaptation. Annual science. Adv. Phys. 66:197–261. Review Of Ecology Evolution And Systematics 44:195– [324] Mehta P, et al. (2018) A high-bias, low-variance intro- 215. duction to Machine Learning for physicists. [303] Mustonen V, LässigM (2010) Fitness flux and ubiq- [325] Dekosky BJ, et al. (2014) In-depth determination and uity of adaptive evolution. Proceedings of the National analysis of the human paired heavy- and light-chain an- Academy of Sciences of the United States of America tibody repertoire. Nat. Med. 21:1–8. 107:4248–4253. [326] Klein AM, et al. (2015) Droplet Barcoding for Single-Cell Transcriptomics Applied to Embryonic Stem 79

Cells Resource Droplet Barcoding for Single-Cell Tran- phocytes. Nat. Protoc. 11:429–442. scriptomics Applied to Embryonic Stem Cells. Cell [328] Grigaityte K, et al. (2017) Single-cell sequencing re- 161:1187–1201. veals αβ chain pairing shapes the T cell repertoire. [327] Mcdaniel JR, DeKosky BJ, Tanno H, Ellington AD, bioRxiv:213462. Georgiou G (2016) Ultra-high-throughput sequencing [329] Howie B, et al. (2015) High-throughput pairing of of the immune receptor repertoire from millions of lym- T cell receptor a and b sequences. Sci. Transl. Med. 7:301ra131.