Signature Redacted May 14,2014 Certified By

System Identification of Cortisol Secretion:

Characterizing Pulsatile Dynamics OF TECHNOLOGy by Rose Taj Faghih JUN 1 0 2014 B.S., Electrical Engineering (2008) University of Maryland, College Park LIBRARIES S.M., Electrical Engineering and Computer Science (2010), Massachusetts Institute of Technology Submitted to the Department of Electrical Engineering and Computer Science in partial fulfillment of the requirements for the degree of Doctor of Philosophy at the MASSACHUSETTS INSTITUTE OF TECHNOLOGY June 2014 @ Massachusetts Institute of Technology 2014. All rights reserved.

Signature redacted Author ...... Department of Electrical Engineering and Computer Science Signature redacted May 14,2014 Certified by...... Emery N. Brown Edward Hood Taplin Professor of Medical Engineering Professor of Computational Neuroscience Thesis Supervisor

Certified by Si onRtIIm mdRctE~d Munther A. Dahleh Professor of Electrical Engineering and Computer Science Professor of Engineering Systems Division Thesis Supervisor b Signature redacted,; A cce;pte d Uy ...... 4, ...... S/Prdsar Leslie A. Kolodziejski Chair, Department Committee on Graduate Students

System Identification of Cortisol Secretion: Characterizing Pulsatile Dynamics by Rose Taj Faghih

Submitted to the Department of Electrical Engineering and Computer Science on May 14, 2014, in partial fulfillment of the requirements for the degree of Doctor of Philosophy

Abstract

Cortisol controls the body's metabolism and response to inflammation and stress. Cortisol is released in pulses from the adrenal glands in response to pulses of adrenocorticotropic hormone (ACTH) released from the anterior pituitary; in return, cortisol has a negative feedback effect on ACTH release. Modeling cortisol secretion and the interactions between ACTH and cortisol allows for quantifying normal and abnormal physiology and can potentially be used for diagnosis and optimal treatment of some cortisol disorders. Due to noise, modeling these interactions using concurrent data from serum ACTH and cortisol levels is challenging. First, using serum cortisol levels, we model cortisol secretion from the adrenal glands by representing the sparse pulses of cortisol using an impulse train. We formulate an optimization problem and successfully recover infusion and clearance rates as well as physiologically plausible cortisol pulses. Then, for serum ACTH and cortisol levels, we model ACTH and cortisol secretion by representing the sparse ACTH pulses using an impulse train. By considering a multi-rate system, we formulate another optimization problem and successfully recover model parameters as well as physiologically plausible ACTH pulses. We solve both optimization problems under the assumption that the number of pulses is between 15 to 22 pulses over 24 hours, and recover the timing and amplitudes of the pulses using compressed sensing, and employ generalized cross validation for determining the number of pulses. In all our studies mentioned above, the datasets we use consist of ACTH and cortisol levels sampled at 10-minute intervals from 10 healthy women. Finally, we present a mathematical characterization of pulsatile cortisol secretion. We hypothesize that there is a controller in the anterior pituitary that leads to pulsatile release of cortisol, and propose a mathematical formulation for such controller. Our proposed controller achieves impulse control, and the obtained impulses and plasma cortisol levels exhibit cortisol circadian and ultradian rhythms that are in agreement with experimental data.

3 Thesis Supervisor: Emery N. Brown Title: Edward Hood Taplin Professor of Medical Engineering Professor of Computational Neuroscience

Thesis Supervisor: Munther A. Dahleh Title: Professor of Electrical Engineering and Computer Science Professor of Engineering Systems Division

4 5 .A a

4)L

'Woudtthat 6eing (existence) fiad a tongue, that it might remove the vei(s from existent beings!

O'6reath of (phenomenaf) existence, whatsoeverywords thou ma-gst utter, know that therefirg thou hast 6ound anotherveiCupon it.

e Mat na-wi of Jacaraddin %uiti TransCation and Commentanj, Nichoson

6 TIo my beI~vedyarents andtn- CoveC-g 6rotir

7 Acknowledgments

I am greatly indebted to my advisors Professor Emery N. Brown and Professor Munther A. Dahleh as this thesis would not have been possible without their guidance, depth of knowledge, understanding, ideas, unwavering support, patience, invaluable advice, and precious feedback. Working under the supervision of these brilliant minds and goodhearted souls, and having them as my role models, has truly been a great honor and privilege for me. I am very lucky that I could benefit from the light of their wisdom in my academic and personal life, and I hope to benefit from their wisdom even more in the years to come. They have been immensely encouraging and great sources of enthusiasm, inspiration, and positive energy. While guiding me with my research, they have always given me freedom to explore research areas and approaches for tackling problems independently to improve my abilities to think deeply and critically to grow as an independent researcher. They have also provided me with the opportunity to collaborate with researchers from different backgrounds to work on an interdisciplinary research project. I would like to express my gratitude to my committee member Professor George Verghese for helpful conversations, insightful suggestions and invaluable advice. I was very fortunate that I could benefit from his support, kindness, and wisdom. He is enormously devoted to students and spent many hours of his precious time giving me great feedback and advice on my research. I am grateful to our collaborators Dr. Elizabeth Klerman and Dr. Gail Adler for providing us with ACTH and cortisol data, which made this research possible. I am also grateful to them for providing me with feedback on Chapters 2 and 3 of this thesis, particularly the discussions concerning the physiological aspects of this research. They have read multiple drafts of these chapters, providing me with great feedback and comments on improving these chapters. From our interactions, I have learned

8 how to bridge the communication gap between electrical engineering and physiology and communicate with medical doctors.

I would like to thank Professor Ketan Savla for mentoring me during my masters and early stages of my PhD when he was a research scientist at MIT. I have learned a lot from Ketan through our collaborations and his kind mentorship; I am grateful for his invaluable advice that has been crucial for me to grow as a researcher.

I was very fortunate to be the head teaching assistant for Professor Babak Ayazi- far, Professor Dimitri Bertsekas, Professor Jeff Shapiro, and Dr. Ligong Wang for 6.041/6.431: Probabilistic Systems Analysis and Applied Probability. I learned a lot from them and their teaching styles. Babak inspires me with his art of teaching; I am indebted to him for the invaluable lessons I learned from him regarding teaching skills. Babak and Dimitri also provided me with great advice on what it takes to manage a big and diverse class of graduate and undergraduate students. I am grateful to my academic advisor Professor Muriel Medard for her kind advice throughout my graduate studies. I would like to thank Professor Terry Orlando for his support and advice. I would also like to thank all the professors at the Laboratory for Information and Decision Systems (LIDS) for creating an intellectually stimulating environment in the lab.

I would like to express my gratitude to my undergraduate professors who had a great influence on my academic career. I would like to thank Dr. Donald Day for introduc- ing me to undergraduate research by nominating me for my first research experience as an undergraduate at Drexel University. I would like to thank Professor Thomas Antonsen, Professor Edward Ott and Professor Michelle Girvan for co-supervising my second research experience as an undergraduate at the University of Maryland. I would like to specially thank Professor Antonsen for being a great mentor, spending a lot of his precious time providing me with advice, which has been crucial to my academic career. I would like to also thank Professor Gilmer Blankenship, Professor Rama Chellappa, Professor Christopher Davis, Professor Satyandra Gupta, Professor

William Levine, and Professor Edo Waks for their advice and support during my undergraduate studies.

9 I would like to thank Sheri Leone and Albert Carter for being super helpful and friendly, as well as their wonderful administrative work. I would like to specially thank Sheri for her friendship. I would like to thank Brian Jones for tech support. I would also like to thank Debbie Wright, Jenifer Donovan, and Janet Fischer for administrative support.

My EECS friends and labmates in LIDS and Neuroscience Statistics Research Lab- oratory (NSRL) have been an important part of my life as a graduate student. I would like to specially thank my lovely EECS friends Audrey Fan, Shreya Saxena, Christina Lee, and Ermin Wei for their friendship and all the enjoyable moments over the years. Working on interdisciplinary research, I had the opportunity to be a member of two amazing laboratories and benefit from the stimulating environments and mentorship of my labmates in both labs. This has exposed me to a wide variety of philosophies and has provided me with a unique opportunity to interact with researchers in different fields. I would like to thank all my friends and colleagues in both LIDS and NSRL. In alphabetical order, I would like to especially thank my current and former labmates: Elie Adam, Amir Ali Ahmadi, Demba Ba, Behtash Babadi, Giancarlo Baldan, Zhe (Sage) Chen, Fransisco Flores, Qingqing Huang, Yola Katsar- gyri, Pavitra Krishnaswamy, Ying Liu, Wasim Malik, Donatello Materassi, Marzieh Parandehgheibi, Michael Rinehart, and Vincent Tan. It has been a great pleasure having such great friends and colleagues. I would like to express my immense gratitude to my family: my amazing parents Leyla and Nezam, and my lovely brother Ali for their unconditional love and support. By always believing in me and my dreams, they lead me to believe I can pursue and accomplish my academic and non-academic goals in life. Words cannot express my gratitude towards my parents. My parents are constant sources of inspiration, my first teachers and my role models; they instilled the passion for learning in me. Thanks to their upbringing and their provision of an intellectual environment that integrated love, knowledge, arts, and literature, I have grown to define home as a place that incorporates these components all together. My mother is my universe and symbol of selfless pure love, and I admire her outstanding patience and infinite

10 kindness to everyone around her. There is art and beauty in everything that she does. When I look at the paintings and art work that she has made, or read the textbooks or the poems she has written, I tell myself, I want to grow to be like her. My father as I wrote in one of my poems is the sky of knowledge to me and I am always inspired by the depth of his knowledge and various books that he has authored ranging from engineering to literature. I would like to grow to be like him, too! I would like to thank my amazing brother and my best friend, Ali, for always being there for me, and his full support and advice. I cherish all our good memories. Ali is not only my brother, but also my colleague at Munther's group in LIDS. Ali has read this thesis and provided me with feedback. I would like to thank my aunt Ladan for all her love and support especially during my undergraduate studies. I would also like to thank my grandparents for warming my heart with their love and always giving me positive energy and encouraging me to go all the way in my academic career. They have been counting days to see my graduation, and regardless of whether they are standing next to me today embracing me with affection or it is their loving memory that is embracing my heart, I cherish their love forever.

11 12

'IIM' Contents

1 Introduction 27

1.1 Endocrine Hormones ...... 27

1.1.1 Cortisol ...... 29

1.1.2 Cortisol Disorders ...... 29

1.2 Thesis Outline ...... 30

1.2.1 Deconvolution of Serum Cortisol Levels by Compressed Sensing 30

1.2.2 Quantifying Pituitary-Adrenal Dynamics and Deconvolution of Concurrent Cortisol and Adrenocorticotropic Hormone Data by Compressed Sensing ...... 31

1.2.3 An Optimization Formulation for Characterization of Pulsatile

Cortisol Secretion ...... 32

1.2.4 Related Publications ...... 32

2 Deconvolution of Serum Cortisol Levels by Compressed Sensing 35

2.1 Introduction ...... 35

2.2 Experiment ...... 38

13 2.3 Modeling Formulation ...... 40

2.4 Model Estimation ...... 42

2.5 R esults ...... 48

2.6 Discussion ...... 56

3 Quantifying Pituitary-Adrenal Dynamics and Deconvolution of Con- current Cortisol and Adrenocorticotropic Hormone Data by Com- pressed Sensing 61

3.1 Introduction ...... 61

3.2 Experiment ...... 64

3.3 Modeling Formulation ...... 65

3.4 Model Estimation ...... 68

3.5 Results ...... 73

3.6 Discussion ...... 80

4 An Optimization Formulation for Characterization of Pulsatile Cor- tisol Secretion 86

4.1 Introduction ...... 86

4.2 Methods ...... 89

4.3 Results ...... 93

4.4 Discussion ...... 98

5 Conclusion and Future Work 104

14 5.1 Conclusion ...... 104

5.1.1 Deconvolution of Serum Cortisol Levels by Compressed Sensing 104

5.1.2 Quantifying Pituitary-Adrenal Dynamics and Deconvolution of Concurrent Cortisol and Adrenocorticotropic Hormone Data by Compressed Sensing ...... 105

5.1.3 An Optimization Formulation for Characterization of Pulsatile

Cortisol Secretion ...... 105

5.2 Future Work ...... 106

15 16 List of Figures

2-1 Twenty-Four-Hour Serum Cortisol Levels in 10 Women. Each panel displays the cortisol level in one of the participants. The cortisol levels are recorded every 10 minutes...... 49

2-2 Estimated Deconvolution of the Experimental Twenty-Four- Hour Cortisol Levels in 10 Women. Each panel shows the measured 24-hour cortisol time series (red stars), the estimated cortisol levels (black curve), the estimated pulse timing and amplitudes (blue vertical lines with dots) for one of the participants. The estimated

model parameters are given in Table 2.2...... 51

2-3 White Gaussian Structure in the Model Residuals of 10 Women. In each panel, (i) the top sub-panel displays the autocorrelation function of the model residuals in one of the 10 participants; the graph shows that the model captures the dynamics and that residuals are white; (ii) the bottom sub-panel displays the quantile-quantile plot of the model residuals for that participant; the graph shows that the residuals are Gaussian...... 52

17 2-4 Simulated Twenty-Four-Hour Cortisol Levels with Measure- ment Errors Corresponding to Datasets from 10 Women. Each panel displays the simulated serum cortisol levels based on pulse patterns in Figure 2-2 and estimated model parameters 01 and 02 in Ta- ble 2.2 in one of the 10 participants, assuming a zero mean Gaussian measurement error with standard deviation o, in Table 2.3. In all sim-

ulations the initial conditions are x1 (0) = 0, x 2 (0) equals the initial cortisol level of the corresponding participant, and the cortisol levels are recorded every 10 minutes...... 53

2-5 Estimated Deconvolution of Simulated Twenty-Four-Hour Cor- tisol Levels with Different Measurement Errors Correspond- ing to Datasets from 10 Women. Each panel shows the simulated 24-hour cortisol time series (blue stars), the estimated cortisol levels (black curve), the simulated pulse timing and amplitudes (blue vertical lines with dots) and the estimated pulse timing and amplitudes (red vertical lines with empty circles) for one of the simulated datasets that each correspond to a participant. The estimated parameters are given in Table 2.3...... 57

3-1 Twenty-Four-Hour Serum ACTH and Cortisol Levels in 10 Women. In each panel, (i) the top sub-panel displays the ACTH level and (ii) the bottom sub-panel displays the cortisol level in the corresponding participant. The ACTH and cortisol levels are recorded every 10 minutes. The shaded gray area corresponds to sleep period and the white area corresponds to wake period...... 74

18 3-2 Estimated Deconvolution of the Experimental Twenty-Four- Hour Concurrent ACTH and Cortisol Levels in 10 Women. In each panel, (i) the top sub-panel shows the measured 24-hour ACTH

time series (red stars), and the estimated ACTH levels (black curve), (ii) the middle sub-panel shows the measured 24-hour cortisol time series (red stars), and the estimated cortisol levels (black curve), (iii) the bottom sub-panel shows the estimated pulse timing and amplitudes (blue vertical lines with dots) using concurrent measurements of ACTH and cortisol for the corresponding participant. The shaded gray area corresponds to sleep period and the white area corresponds to wake period. The estimated model parameters are given in Table 3.1. . .. 75

3-3 White Gaussian Structure in the Model Residual Errors of ACTH Levels in 10 Women. In each panel, (i) the top sub-panel

displays the autocorrelation function of the ACTH model residual errors in one of the 10 participants; the graph shows that the model

captures the dynamics and that ACTH residual errors are white; (ii) the bottom sub-panel displays the quantile-quantile plot of the ACTH model residual errors for that participant; the graph shows that the

ACTH residual errors are Gaussian...... 77

3-4 White Gaussian Structure in the Model Residual Errors of Cortisol Levels in 10 Women. In each panel, (i) the top sub-panel displays the autocorrelation function of the cortisol model residual errors in one of the 10 participants; the graph shows that the model captures the dynamics and that cortisol residual errors are white; (ii) the bottom sub-panel displays the quantile-quantile plot of the cortisol model residual errors for that participant; the graph shows that the cortisol residuals are Gaussian ...... 78

19 3-5 Comparison of Estimated Pulse Timing and Amplitudes Us- ing the Experimental Twenty-Four-Hour Cortisol Levels Only with Estimates from Concurrent ACTH and Cortisol Levels in 10 Women. Each panel shows the estimated pulse timing and amplitudes (blue vertical lines with dots) using concurrent measurements of ACTH and cortisol, and the estimated pulse timing and amplitudes (red vertical lines with dots) using only cortisol measurements for the

corresponding participant. The shaded gray area corresponds to sleep period and the white area corresponds to wake period...... 81

4-1 Cortisol Levels and Control Obtained Using Example 1 (i) The top panel displays the optimal cortisol profile (black curve), constant upper bound (red curve), and constant lower bound (blue curve). (ii) The bottom panel displays the optimal control. The optimization problem obtained 12 impulses over 24 hours as the optimal control (the timing of the control was discretized into 1440 points; the obtained control takes 12 nonzero values, i.e. impulses, while it is zero everywhere else). The optimization problem was solved using the parameters given in Example 1 in Table 4.1 and the upper and lower bounds provided in Tables 4.2 and 4.3, respectively ...... 94

20 4-2 Cortisol Levels and Control Obtained Using Example 2 (i) The top panel displays the optimal cortisol profile (black curve), two- harmonic upper bound (red curve), and two-harmonic lower bound (blue curve). (ii) The bottom panel displays the optimal control. The optimization problem obtained 16 impulses over 24 hours as the optimal control (the timing of the control was discretized into 1440 points; the obtained control takes 16 nonzero values, i.e. impulses, while it is zero everywhere else). The optimization problem was solved using the parameters given in Example 2 in Table 4.1 and the upper and lower bounds provided in Tables 4.2 and 4.3, respectively...... 96

4-3 Cortisol Levels and Control Obtained Using Example 3 (i) The top panel displays the obtained cortisol profile (black curve), two- harmonic upper bound (red curve), and two-harmonic lower bound (blue curve). (ii) The bottom panel displays the obtained control. The optimization problem obtained 16 impulses over 24 hours as the control (the timing of the control was discretized into 1440 points; the obtained control takes 16 nonzero values, i.e. impulses, while it is zero everywhere else). The optimization problem was solved using the parameters given in Example 3 in Table 4.1 and the upper and lower

bounds provided in Tables 4.2 and 4.3, respectively...... 97

21 4-4 Cortisol Levels and Control Obtained Using Example 4 (i) The top panel displays the obtained cortisol profile (black curve), two- harmonic upper bound (red curve), and two-harmonic lower bound (blue curve). (ii) The bottom panel displays the obtained control. The optimization problem obtained 12 impulses over 24 hours as the control (the timing of the control was discretized into 1440 points; the obtained control takes 12 nonzero values, i.e. impulses, while it is zero everywhere else). The optimization problem was solved using the parameters given in Example 4 in Table 4.1 and the upper and lower bounds provided in Tables 4.2 and 4.3, respectively...... 99

22 23 List of Tables

2.1 Clinical Characteristics of the Participants ...... 39

2.2 The Estimated Model Parameters and the Squares of the Mul- tiple Correlation Coefficients (R 2 ) for the Fits to the Experi- mental Cortisol Time Series...... 48

2.3 The Estimated Model Parameters and the Squares of the Mul- tiple Correlation Coefficients (R2 ) for the Fits to the Simu- lated Cortisol Time Series ...... 54

2.4 The Error in Estimated Pulses for the Simulated Cortisol Time Series ...... 55

3.1 The Estimated Model Parameters for the Fits to the Experi- mental ACTH and Cortisol Time Series ...... 76

3.2 The Estimated Number of Pulses and the Squares of the Mul- tiple Correlation Coefficients (R2) for the Fits to the Experi- mental ACTH and Cortisol Time Series ...... 79

4.1 Model Parameters for Examples of Optimization Problem (4.1) ...... 92

4.2 Upper Bounds for Examples of Optimization Problem (4.1) . 92

24 4.3 Lower Bounds for Examples of Optimization Problem (4.1) . 93

25 26 Chapter 1

Introduction

1.1 Endocrine Hormones

Hormones are chemical messengers that are released from the endocrine glands into the circulation, and relay information to cells and control a wide range of physio- logic functions [22]. The endocrine system consists of several glands that produce hormones, and secretion of endocrine hormones is controlled in a hierarchical manner. For some hormones, neural interactions in the hypothalamus result in release of hormone-releasing hormones from the hypothalamus. Then, hormone-releasing hormones induce release of pituitary hormones from the pituitary, and the pituitary hormones induce secretion of hormones from target glands. These hormones, which are absorbed from the blood, implement regulatory functions in different parts of the body and travel back in the blood to the pituitary and the hypothalamus and have a feedback effect on release of hormone-releasing hormones and pituitary hormones. A key aspect of endocrine function is the feedback control of hormone production and secretion, and feedback loops have been identified for virtually all hormone systems [22].

Secretion of most endocrine hormones is driven by a similar control mechanism. Un-

27 derstanding how one endocrine hormone is released adds insight to how the rest of the endocrine hormones are released. As a prototype, in this thesis, we study the hypothalamic-pituitary-adrenal (HPA) axis. A similar control feedback system underlies the release of growth hormone (GH), thyroid hormone, estrogen, and testosterone. One of the interesting features of these control systems is the pulsatile mechanism in release of these hormones. When the endocrine system is functioning normally in a healthy person, the plasma is a time-varying function with regular periodic patterns. For example, the 24-hour plasma cortisol profile consists of episodic release of 15 to 22 secretory events with varying amplitudes in a regular diurnal pattern, with the lowest amplitude occurring between 8 PM and 2 AM, increasing rapidly throughout the late night, with the highest amplitude between 8 AM and 10 AM, and then, the amplitude declines throughout the day [4].

Endocrine disorders may exist in the form of endocrine gland hyposecretion (hormone deficiency), endocrine gland hypersecretion (hormone excess), or tumors of endocrine glands. Such disorders are treated using surgery, tablets, or injections. The dosage used in the current medication methods is not optimal and may cause other disorders. However, a desired treatment should use an optimal dosage (amount and timing) by employing a model that predicts the dose-response. Since there are many endocrine disorders that affect the patient's performance in various ways, it is important to have a model for hormone secretion to potentially be able to use an optimal approach in treating hormonal disorders to minimize the side-effects of the medication. Different hormonal disorders motivate this study; for example, normal endocrine secretion is necessary for cardiovascular health, and the cardiovascular system benefits from cor- recting endocrine disorders [34].

28 1.1.1 Cortisol

Cortisol is a steroid hormone that regulates the metabolism and the body's reaction to stress and inflammation [4]. Stress can be physical, such as infection, thermal exposure, and dehydration, or psychological, such as fear and anticipation [16]. Cor- tisol relays rhythmic signals from the suprachiasmatic nucleus (SCN), the circadian pacemaker, to synchronize bodily systems with environmental variations [37]. In the hypothalamus, SCN sends a harmonic circadian signal to the paraventricular nu- clei (PVN), which leads to release of corticotropin releasing hormone (CRH). CR1H secreted into the hypophyseal portal blood vessels induces release of Adrenocorti- cotropic hormone (ACTH) from the anterior pituitary [4]. ACTH is synthesized and stored in the anterior pituitary gland. Synthesis and release can occur independently, and are stimulated by CRH [7]. Then, via stimulation of adrenal glands by ACTH, adrenal glands produce and secrete cortisol [24], [4]. After synthesis, cortisol diffuses into the circulation and is absorbed from the blood plasma by different tissues where it implements regulatory functions as a steroid hormone. Then, cortisol is cleared from the plasma by the liver [4]. Moreover, cortisol has a feedback effect on the hypothalamus and pituitary as well as CRH and ACTH secretion [16], [24], [4]. It is also known that PVN produces a corticotropin-inhibiting factor, which inhibits ACTH synthesis and release [31].

1.1.2 Cortisol Disorders

One instance of a cortisol disorder is adrenal deficiencies that might be due to impairment of the adrenal glands, impairment of the pituitary gland or the hypothalamus. An example of a disorder caused by adrenal deficiency is Addison's disease. Persistent vomiting, anorexia, hypoglycemia, poor weight gain in a child, or unexplained weight loss in an adult, fatigue, muscular weakness, and unexplained dehydration can be caused by adrenal deficiency [41]. Injections or tablets are used in treating cortisol

29 deficiencies.

Adrenal hormone excess is another disorder of the HPA axis. An example of adrenal excess is Cushing's syndrome, in which the cortisol level in the blood is high. Cush- ing's syndrome results in weight gain, central obesity, fatigue, muscle weakness, hyper- tension, diabetes, acne, and menstrual disorders [34]. Untreated Cushing's syndrome can cause heart disease. Treatment options for this disorder are transsphenoidal surgery, unilateral or bilateral adrenalectomy, radiotherapy and medical therapy [34]. Cushing's syndrome can be caused by taking glucocorticoid drugs. For example, steroid treatment of disorders such as asthma can cause Cushing's syndrome [19].

1.2 Thesis Outline

This thesis investigates three problems related to the HPA axis. The first part of this thesis recovers the infusion and clearance rates and cortisol pulses from serial measurements of serum cortisol levels. The second part of this thesis models the interactions between ACTH and cortisol, and recovers the infusion and clearance coefficients, feed-forward and feedback gains, and ACTH pulses. Finally, the last part of this thesis proposes a plausible optimal control characterization of cortisol secretion.

1.2.1 Deconvolution of Serum Cortisol Levels by Compressed

Sensing

Determining the number, timing, and amplitude of cortisol secretory events and recovering the infusion and clearance rates from serial measurements of serum cortisol levels is a challenging problem. Despite many years of work on this problem, a

30 complete satisfactory solution has been elusive. We formulate this question as a non- convex optimization problem, and solve it using a coordinate descent algorithm that has a principled combination of (i) compressed sensing for recovering the amplitude and timing of the secretory events, and (ii) generalized cross validation for choosing the regularization parameter. Using only the observed serum cortisol levels, we model cortisol secretion from the adrenal glands using a second-order linear differential equation with pulsatile inputs that represent cortisol pulses released in response to pulses of ACTH. Then, we analyze simulated datasets and actual 24-hr serum cortisol datasets sampled every 10 minutes from 10 healthy women. Identification of the amplitude and timing of pulsatile hormone release allows for (i) quantifying normal and abnormal secretion patterns to understand pathological neuroendocrine states, and (ii) potentially designing optimal approaches for treating hormonal disorders.

1.2.2 Quantifying Pituitary-Adrenal Dynamics and Decon- volution of Concurrent Cortisol and Adrenocorticotropic Hormone Data by Compressed Sensing

Interactions of ACTH and cortisol depend on their infusion and clearance coefficients, feed-forward and feedback gains, and the number, timing, and amplitudes of ACTH pulses released in response to CRH. Modeling these interactions, inferring the ACTH pulses, and recovering model parameters using concurrent data from serum ACTH and cortisol levels is a complex problem. For serum ACTH and cortisol levels sampled at 10-minute intervals from 10 healthy women, we model ACTH and cortisol secretion using a system of linear differential equations, and represent the sparse ACTH pulses using an impulse train. Then, by considering a multi-rate system in which the pulsatile input is piecewise constant over one-minute intervals, we formulate an optimization problem. We recover the timing and amplitudes of pulses using compressed sensing, and employ generalized cross validation for determining the number of pulses.

31 1.2.3 An Optimization Formulation for Characterization of Pulsatile Cortisol Secretion

While it is well-known that some hormones such as cortisol are released in pulses, it is not known why it is optimal for the body to release hormones in pulses as opposed to releasing them in a continuous manner. Since pulsatile cortisol release relays distinct signaling information to target cells, and some diseases are linked to changes in cortisol pulsatility, it is crucial to understand the physiology underlying pulsatile cortisol release. Previous studies suggest that a sub-hypothalamic pituitary-adrenal system results in the pulsatile ultradian pattern underlying cortisol release. We hypothesize that there is a controller in the anterior pituitary that leads to pulsatile release of cortisol, and propose a mathematical formulation for such controller, which leads to impulse control as opposed to continuous control. We postulate that this controller is minimizing the number of secretory events that result in cortisol secretion, which is a way of minimizing the energy required for cortisol secretion; this controller maintains the plasma cortisol levels within a specific circadian range while complying with the first order dynamics underlying cortisol secretion. We use an to-norm cost function for this controller, and solve a reweighed f1 -norm minimization algorithm for obtaining the solution to this controller.

1.2.4 Related Publications

The material presented in this thesis are published/in preparation for publication in the following papers: Chapter 2: Deconvolution of Serum Cortisol Levels by Compressed Sens- ing

32 Faghih R.T., Dahleh, M.A., Adler G.K., Klerman E.B., and Brown E.N., Decon- volution of Serum Cortisol Levels by Using Compressed Sensing, PLOS ONE 9(1): e85204, 2014. Chapter 3: Quantifying Pituitary-Adrenal Dynamics and Deconvolution of Concurrent Cortisol and Adrenocorticotropic Hormone Data by Com- pressed Sensing Faghih R.T., Dahleh, M.A., Adler G.K., Klerman E.B., and Brown E.N., Quantifying Pituitary-Adrenal Dynamics and Deconvolution of Concurrent Cortisol and Adreno- corticotropic Hormone Data by Compressed Sensing, in preparation, 2014. Chapter 4: An Optimization Formulation for Characterization of Pulsatile Cortisol Secretion Faghih R.T., Dahleh, M.A., and Brown E.N., An Optimization Formulation for Char- acterization of Pulsatile Cortisol Secretion, in preparation, 2014.

33 34 Chapter 2

Deconvolution of Serum Cortisol Levels by Compressed Sensing

2.1. Introduction

Since many endocrine hormones such as cortisol are released in pulses instead of having slowly varying concentrations, understanding normal and abnormal endocrine functioning requires detection and quantification of both timing and amplitude of hormone pulses [47]. Determining the number, timing, and amplitude of hormone pulses is a challenging problem. For hormones that are released under control of the hypothalamus and anterior pituitary, release is regulated hierarchically; hypothalamic hormone release induces pituitary hormone release [22]. Then, either the pituitary hormone induces the release of another hormone from an endocrine gland (e.g. cortisol from the adrenal glands or thyroid from the thyroid gland) or the hormone of interest

Chapter adopted from Faghih R.T., Dahleh, M.A., Adler G.K., Klerman E.B., and Brown E.N., Deconvolution of Serum Cortisol Levels by Using Compressed Sensing, PLOS ONE 9(1): e85204, 2014.

35 is directly released from the pituitary (e.g. growth hormone). These hormones are absorbed from the blood stream, and implement regulatory functions throughout the body. Some hormones exert negative feedback on release of their hypothalamic and pituitary regulatory hormones, and consequently their further release [22].

Current data analysis methods for pulsatile hormone secretion either assume that the timing of the impulses belongs to a certain class of stochastic processes (e.g. birth-death process) [20] or use pulse detection algorithms [46]. The problem of recovering the model parameters as well as the number, timing, and amplitude of hormone pulses from a limited number of observations is ill-posed (i.e., there could be multiple solutions). However, by taking advantage of the sparse nature of hormone pulses and adding more constraints, the problem becomes more tractable. Veldhuis et al. [451 and Keenan et al. [21] have recently reviewed various methods used for analyzing pulsatile hormone secretion. One method used for analyzing hormones is to assume point process models for the secretory events and employ methods such as the Markov Chain Monte Carlo (MCMC) algorithm [20]. For example, Johnson et al. find hormone pulses by embedding a birth-death process in an MCMC algorithm [20]. Keenan et al. propose an algorithm that uses a Bayesian approach to identify the pulses, and the secretion and clearance rates [21]. These methods assume that the occurrence of the secretory events is stochastic. In a recent work, Vidal et al. [46] use a pulse detection algorithm and remove peaks whose heights are small compared to the other detected pulses or some threshold. This pulse detection algorithm is implemented on luteinizing hormone (LH) data from ewes. In the LH data from ewes, one pulse decays significantly before the next pulse occurs. GH time series patterns also consist of hormone pulses that decay significantly before the next pulse occurs. How- ever, one cortisol pulse can occur before the previous one has significantly decayed to near zero. Therefore, the extraction of the pattern of pulses may be less clear in cortisol than in LH and GH data, and analyzing cortisol data is more challenging.

36 As a first step in the study of novel methods of quantifying pulsatile activity of hormones, we investigate cortisol secretion in the HPA axis. Glucocorticoids, including cortisol, are crucial in neurogenesis, glucose homeostasis, metabolism, stress response, cognition, and response to inflammation [36]. Diseases that are linked to abnormalities in the HPA axis include diabetes, visceral obesity and osteoporo- sis, life-threatening adrenal crises and disturbed memory formation [12], [6]. Cortisol secretion is initiated by the release of CRH from the hypothalamus; CRH induces pulsatile release of ACTH from the anterior pituitary, and through their stimulation by ACTH, the adrenal glands produce cortisol. Cortisol exerts negative feedback effect on the release of CRH and ACTH, and consequently future cortisol release. Normal physiology includes variation of the cortisol level over a 24-hour period; these variations are due to the circadian modulation of the amplitude of the secretory events and ultradian modulation of the timing of the secretory events. Thus, there is a complex physiology required to produce the cortisol pulses and in particular this physiology depends critically on the release of CRH from the hypothalamus and pulsatile release of ACTH from the anterior pituitary. In this chapter, we are only concerned with the problem of estimating the pharmacokinetics properties of cortisol as well as the number, timing and amplitude of the cortisol pulses from the time-series of the serum cortisol levels. Over 24 hours, there are 15 to 22 secretory events [4], [43] with an average of 18 secretory events in both males and females [35]. Hence, since there are a small number of secretory events that are significant, these hormone pulses can be considered sparse and specific analytic techniques can be applied.

In this chapter, we use the characteristic of the sparsity of hormone pulses (i.e., there are a small number of secretory events that are important) and recover the timing and amplitude of individual hormone pulses using compressed sensing techniques. Com- pressed sensing is a technique for perfect reconstruction of sparse and compressible signals using fewer measurements than required by the Shannon/Nyquist sampling theorem [3]. For compressible signals, where only a small number of coefficients are

37 large (i.e., most coefficients are small or zero) and small coefficients can be discarded, the signal can be approximated by a sparse representation and recovered using optimization or greedy algorithms [3]. We describe a coordinate descent approach to recover cortisol secretory events and model parameters. To demonstrate the performance of the algorithm, we apply it to cortisol data. Although we know the sparsity range of the input, the number of pulses for each participant is an open question. In finding the number of pulses, there is a trade-off between capturing the residual error and the sparsity. We use generalized cross-validation to find the number of pulses such that there is a balance between the residual error and the sparsity. This algorithm potentially can be applied to other pulsatile endocrine hormones such as GH, thyroid hormone, LH, and testosterone.

2.2 Experiment

Blood from 10 healthy women collected every 10 minutes for 24 hours was assayed for cortisol in duplicate. The 24 hours began with 8 hours of scheduled sleep followed by 16 hours of wake. The participants were recruited via advertisements in local newspapers to serve as healthy controls for a study on women with fibromyalgia; participants with abnormal laboratory test results or current medical problems were excluded [23]. None of the participants had received glucocorticoids or estrogen/progesterone within the year or 4 months before the study, respectively [23]. For 3 consecutive nights, at the General Clinical Research Center of the Brigham and Womens Hospital, participants had 8 hours of scheduled sleep in the dark at their habitual sleep-wake times and three meals and two snacks; on day 4, all participants started a constant routine protocol that included continuous wake, constant posture, and small meals given hourly [23]. Blood drawing for hormones analyzed in this study was performed during the third night of sleep and during the first 16 hours of the constant routine. A detailed description of the experiment is in [23]; clinical

38 characteristics of the participants are given in Table 2.1. The experimental protocol was designed to minimize the effects of ambulatory temperature, activity, eating, and stress on the participants. Therefore, this dataset can be used to quantify cortisol variations as a function of the time of the day, as controlled by the circadian and the ultradian patterns.

Table 2.1: Clinical Characteristics of the Participants

AGE BMI (' )I Systolic BP (mmHg) Diastolic BP (mmHg) 28 20.8 100 70 41 23.6 120 80 41 22.5 130 70 24 20.7 108 78 23 27.4 108 68 26 25.2 108 68 23 29.3 110 74 37 29.6 138 80 44 29.9 135 82 42 22.9 108 62

BMI and BP refer to Body Mass Index and Blood Pressure, respectively. None of the participants had a current diagnosis of depression.

This project has been reviewed and approved by the Brigham and Women's Hos- pital Institutional Review Board (IRB). During the review of this project, the IRB specifically considered (i) the risks and anticipated benefits, if any, to subjects; (ii) the selection of subjects; (iii) the procedures for securing and documenting informed consent; (iv) the safety of subjects; and (v) the privacy of subjects and confidentiality of the data. All subjects provided written informed consent that was approved by the ethics committees.

39 2.3 Modeling Formulation

We build a model based on the stochastic differential equation model of diurnal cortisol patterns [4]. This model is based on the first-order kinetics for cortisol synthesis in the adrenal glands, cortisol infusion to the blood, and cortisol clearance by the liver, while considering a doubly stochastic pulsatile input in the adrenal glands that has Gaussian amplitudes and gamma distributed interarrival times [4]. This input can be considered as an abstraction of hormone pulses and marks the timing and amplitude of the secretory events leading to cortisol secretion. We assume that there are between 15 and 22 secretory events over a 24-hour period that result in the observed cortisol profile [4, 43, 9, 8]. This model is represented as follows:

dxt = -0 1x1(t) + u(t) (Adrenal Glands) (2.1) dt

dx2 (t) = Oix1(t) - 62 x 2 (t) (Serum) (2.2) dt

where x1 is the cortisol concentration in the adrenal glands and x 2 is the serum cortisol concentration. 01 and 02, respectively, represent the infusion rate of cortisol from the adrenal glands into the blood and the clearance rate of cortisol by the liver. u(t) = qjjg6(t - -) is an abstraction of the hormone pulses that result in cortisol secretion where qj represents the amount of the hormone pulse initiated at time ri

(qi is zero if a hormone pulse did not occur at time i-i), and we assume that impulses occur at integer minute values. N corresponds to the length of the input (N = 1440).

Blood was collected, beginning at yo and then, with a sampling interval of 10 minutes, for M samples (M = 144). All samples were assayed for cortisol. Let yt.o, Yt 20 ,---,

YthoM

Ytk = X2(tk) + vk (2.3) where yt, and vt, represent the observed serum cortisol level and the measurement

40 error, respectively, and missing data points can be interpolated. For each participant, blood samples were assayed in duplicate; hence, for each participant, we could obtain the standard deviation of noise and model the corresponding measurement error. A Gaussian density is a reasonably good approximation of the probability density of the immunoassay error [4], and by using a least squares approach in our estimation algorithm, we model the noise as a Gaussian random variable. Using the serum cortisol level (x 2 ) with a sampling interval of 10 minutes, we would like to estimate 01 and 02, and obtain the number of pulses, their timing, and their amplitude.

Considering the known physiology of de novo cortisol synthesis (i.e., no cortisol is stored in the adrenal glands) [4], we assume that the initial condition of the cortisol level in the adrenal glands is zero (xi(0) = 0) [4], and solve for ytk. Considering that we have discrete data points sampled every 10 minutes, assuming that the input occurs at integer minutes, and yo is the initial condition of the serum cortisol concentration, every observed data point yt, can be represented as follows:

ytk = atkyo + btku + Vtk (2.4)

9 0 where bNk = [eio (e- 2k - e- 0ik) 611 (e- 2(k-) - e- 1(k-1)) ... 910 (e-2 - -91) -- 0 N-k 0 at, = e- 2k, and u represents the entire input over 24 hours (elements of u take values qj for i= 1, ... , 1440).

Let y = [YtIO Yt - YthOM] 20 0 = 0 02] Ao = [at at20 - - atlom

Bo = bio bt2 - btlom , and v = [Vt1 0 Vt 20 - -Vt].

We can represent this system as:

41 y = AOyo+Bou+v. (2.5)

2.4 Model Estimation

In modeling cortisol secretion over 24 hours, results from previous studies suggest that there are 15 to 22 secretory events [4, 43, 9, 8], and on average, there are 18 secretory events [35]. Hence, we assume u contains 15 to 22 nonzero elements out of 1440 possibilities and all these nonzero elements are nonnegative (15 < I uIo l 22, u > 0). To estimate the model parameters, we follow [4] and assume that the infusion rate of cortisol from the adrenal glands to the circulation is at least four times the clearance rate of cortisol by the liver (i.e., 462 < 01). Because the hormone infusion and clearance rates cannot be negative, we further assume in the optimization algorithm that 0 > 0. We can formulate this problem as an optimization problem:

min ||y - Aoyo - Bou12 (2.6)

S..

15 < iiullo < 22 U > 0 CO < b

where C= ,Ob= [0 0 0]'. 4 0 -1

This optimization problem is generally considered as NP-hard. It is possible to use an 4-norm relaxation and solve this problem using different computational strategies such as the basis pursuit, greedy algorithms, iterative-thresholding algorithms, or the FOCUSS algorithm and its extensions [53]. We solve this problem with an extension

42 of the FOCUSS algorithm by first casting this optimization problem as:

min JA(6, u) = illy - Aoyo - BeuI2 + AIllpu, (2.7) CG~b

where the i,-norm is an approximation to the to-norm (0 < p 5 2) and A is chosen such that the sparsity of u is between 15 to 22. Then, by using a coordinate descent approach, this optimization problem can be solved iteratively through the following steps (for 1 = 0, 1,2, ... ) until convergence is achieved:

u(+ = argmin JA(00) , u) (2.8)

60(+1) = argmin JA(O, u('+')) (2.9) CO

The optimization problem in (2.8) now can be solved using the FOCUSS algorithm, which uses a re-weighted norm minimization approach. The solution at each iteration is found by minimizing the f2 -norm and the iteration refines the initial estimate to the final localized energy solution [141. In the FOCUSS algorithm, assuming that a gradient factorization exists, the stationary points of (2.8) satisfy u = PuBT(BePuB + AI)-'yo [14], where P. =diag(ju; 2-P), and ye = y - Aoyo. By iteratively updating A and u until convergence, we can solve for the sparse vector u. In the optimization problem in (2.8), A balances between the sparsity of u and the residual error |lye - Boul|2 . The sparsity of u increases with A.

A version of the FOCUSS algorithm called FOCUSS+ proposed by Murray [30] allows for solving for u such that the maximum sparsity of u is n (n = 22 for our current problem) and u is nonnegative. This algorithm uses a heuristic approach for updating A, which tunes the trade-off between the sparsity and the residual error by increasing

43 A to a maximum regularization Ama,-as the residual error decreases. FOCUSS+ works as follows:

1. P (r =diag(l U r 2-p)

2. A(r) = (1 _ I jyBoU(r)I2)Amax, A > 0

3. U('+) = p() T(BoP(r)BT + A(r)I)-lya

4. Ur+l < 0 _+ Ur+l) = 0

5. After more than half of the selected number of iterations, if j|U(r+l)11o > n, select the largest n elements of U(r+l) and set the rest to zero.

6. Iterate

FOCUSS+ usually converges in 10 to 50 iterations [30]. Although an estimate of the unknown quantities can be obtained after iteratively solving for u and 0 (2.8-2.9) using FOCUSS+, 0 and u should be updated by finding an optimal choice of A such that enough noise is filtered out and the estimated u is not capturing residual error by finding a less sparse solution. We use the Generalized Cross-Validation (GCV) technique [13] for estimating the regularization parameter such that there is a balance in filtering out the noise and the sparsity of u. The GCV function is defined as:

G(A) = L11(I - HA)ye|12 2 (2.10) (trace(I- H)) ' where L is the number of data points, and HA is the influence matrix. For the FO- CUSS algorithm, H\ = BePuBOT(BoPuBO + AI)- 1. Zdunek et al. [53] employ the GCV technique for estimating the regularization parameter for the FOCUSS algorithm through singular value decomposition:

L Ii2( A)2 G(A) = , (2.11) ( L=g )2

44 where C = RTy9 = CL]-and B9 P = REQT with E = diag{a}; R and Q are unitary matrices and o-'s are the singular values of BePu [53]. Zdunek et al. [53] propose minimizing G(A) such that A is bounded between some minimum and maximum values (Amin and Am.) using the MATLAB function fminbnd (MATLAB R2011b), which is an implementation of the golden section (GS) search. Although the GS search only finds a local extremum, considering that G(A) is unimodal, the GS search always finds the desired solution given a large range for A [53]. We used a range of zero to 10 for A.

Considering the ill-posedness of deconvolution problems, small variations in the data can result in large changes in the solution, and a balanced choice of regularization is required to filter out the effect of noise. Tikhonov regularization, truncated singular value decomposition, and the method of L-curve are well-known methods used when dealing with such problems [17]; among these methods, the L-curve method appears to be the most commonly used. Automatically searching for the minimum of the GCV function is easier than finding the corner of the L-curve as the L-curve method is computationally expensive, requiring computation of the solution for several samples of the parameter [53]. Moreover, Zdunek et al. point out that GCV is usually more accurate in estimating the regularization parameter than the L-curve method [53]. In the GCV technique, the optimal choice of regularization minimizes the pre- dictive mean-squared error. Hence, one can update u by modifying FOCUSS+ using the GCV method. For r = 0, 1, 2, ... GCV-FOCUSS+ works as follows:

1. pr) =djag(IU4r12-p)

2. U(r+l) = p(r)BT(B p(r)BT + A(r)I)-ly,

3. u r+' < 0 = 0

4. A(r+) = argmin G(A) O

45 5. Iterate until convergence

By combining the GCV method with FOCUSS+, one can find an optimal choice of A at each iteration such that enough noise is filtered out when solving for u, and iterate between solving (2.8) and (2.9) until convergence is achieved. The following is the algorithm that we propose for deconvolution of cortisol data:

1. Initialize 90 by sampling a uniform random variable w on [0, 1], and let 90 =

[ i'and j = 1

2. Set 9 equal to 6i-1; using FOCUSS+, solve for I by initializing the optimization problem in (2.8) at a vector of all ones

3. Set 5 equal to 6i; using the interior point method, solve for 93 by initializing the optimization problem in (2.9) at & ~

4. Iterate between steps 2-3 for 30 iterations

5. Initialize 90 and 60 by setting them equal to the 9i and u* that minimize JA(9, u) in (2.7), and let i = 1

6. Set d equal to 6i-1; using GCV-FOCUSS+, solve for bi by initializing the optimization problem in (2.8) at 0t--

7. Set 6 equal to Wt; using the interior point method, solve for 05 by initializing the optimization problem in (2.9) at 9i~

8. Iterate between steps 6-7 until convergence

9. Repeat steps 1-8 for various initializations

10. Set the estimated model parameters 9 and input 6 equal to the values that minimize JA(0, u) in (2.7). Since this optimization problem is non-convex, there are multiple local minima, and a reasonable procedure to choose among the local minima is to select the one with the best goodness of fit.

46 Step 1 initializes the algorithm randomly. Steps 2-4 use the random initialization to find a good initialization for the unknowns while using FOCUSS+ for sparse recovery and interior point method for finding the model parameters. Step 5 finds a good initial condition by comparing the estimates obtained in Steps 2-4, and selecting the 0 and u values that minimize the cost function. The values found at Step 5 are used for initializing the main algorithm. Steps 6-8 use a coordinate descent approach to estimate the unknowns until convergence is achieved. Sparse recovery is achieved using GCV-FOCUSS+ which uses generalized cross-validation for finding the regularization parameter; GCV-FOCUSS+ selects the regularization parameter such that there is a balance between capturing the sparsity and the noise, and finds different sparsity levels for different individuals. Step 9 repeats the initialization and estimation steps for various initializations. Step 10 selects the 0 and u values that minimize the cost function.

The main idea behind the algorithm is to solve the non-convex problem in (2.7) using a coordinate descent approach to converge to a local minimum for different initializations, and choose the local minimum that minimizes the problem in (2.7). Convergence properties of coordinate descent algorithms are well-studied and a discussion can be found in [1].

We have implemented the algorithm by assuming that hormone pulses occur at integer minutes. We ran the proposed algorithm using 10 random initializations for each dataset. According to [18], the FOCUSS algorithm converges faster for 0 < p < 1 compared to 1 < p < 2; however, it should be noted that for 0 < p < 1, the optimization problem is not convex and if p is too small (e.g., p = 0.1), it is possible to stagnate into a local minimum; hence, they suggest selecting a value slightly smaller than 1 but not too small [18]. When running GCV-FOCUSS+, we let p = 0.5 to solve for u. Data analysis, estimation, and simulations were performed in MATLAB R2011b.

47 Using the cortisol secretion model (2.1-2.2), we simulated ten 24-hour cortisol datasets using parameters 01 and 02 in Table 2.2 and impulse trains in Figure 2-2. Then, using the observation model (2.3), we added zero mean Gaussian noise with a standard deviation of a-, to the simulated cortisol levels; o-, in Table 2.3 models the immunoassay error for each participant and was obtained using blood samples assayed in duplicate.

Table 2.2: The Estimated Model Parameters and the Squares of the Multi- ple Correlation Coefficients (R 2 ) for the Fits to the Experimental Cortisol Time Series.

1 0.0739 0.0067 18 0.97 2 0.0762 0.0057 17 0.93 3 0.0921 0.0082 16 0.96 4 0.1248 0.0061 17 0.93 5 0.0585 0.0122 18 0.95 6 0.0726 0.0095 20 0.96 7 0.0799 0.0107 16 0.97 8 0.0365 0.0091 16 0.93 9 0.0361 0.0090 16 0.92 10 0.0864 0.0073 20 0.94 Median 0.0751 0.0086 17 0.95

The parameters 01 and 02 are, respectively, the estimated infusion rate of cortisol into the circulation from the adrenal glands and the estimated clearance rate of cortisol by the liver. N is the estimated number of hormone pulses and R2 is the square of the multiple correlation coefficient.

2.5 Results

Figure 2-2 shows experimental data from each participant. For most participants, the cortisol level is low at the beginning of the scheduled sleep; the cortisol level increases rapidly around the wake time and gradually decreases throughout the day.

48 P "pwp=4 Pmft*" 2f mms vwm.

S4 a I to It . Is 1$ t 2 S4 6 1 e W 4 14 10 18 20 22 24 rae m 4 P rse pan 3

mmm4

It 12 14 If 2f 22 24 2. s 4 S 1214t It n 2

i 0 2 1 s n4 nS aS 4 2 4 s-0 It12 1- 1-1i 26 22 34 Tmm-w hmm PuMs~puiS

I *1 4 is 26 2224 I

P 16 M-ON-0 - 2 g24 I IU

161 4 1 2& 22 24 TmP a IsIt 4 i s2 22

Figure 2-1: Twenty-Four-Hour Serum Cortisol Levels in 10 Women. Each panel displays the cortisol level in one of the participants. The cortisol levels axe recorded every 10 minutes.

49 Figure 2-2 shows the estimated amplitude and timing of hormone pulses, experimental cortisol data, and model-predicted cortisol estimates for each participant. There are variations in the timing and amplitudes of the detected hormone pulses. The circadian amplitudes of the recovered pulses demonstrate the known circadian variation of cortisol time series [4]; for most participants, the recovered pulses are small at the beginning of the scheduled sleep, and there is a large pulse towards the end of the sleep period or beginning of the wake period. There are multiple small and medium sized pulses during the wake period. The number of detected pulses for all participants are within their corresponding physiologically plausible ranges [4, 43, 9, 8] with a square of the multiple correlation coefficient (R2 ) above 0.92 (Table 2.2). The R2 is a statistical measure of the goodness of fit obtained by model-predicted estimates of data; it measures the fraction of the sample variance of data that is predicted by the model: R2 values close to 1.0 suggest that the model is good at estimating the data. Figure 2-3 shows the autocorrelation function and the quantile-quantile plots of the model residuals for the 10 participants suggesting that the model captures the dynamics, and that the residuals have a Gaussian structure and are white.

We simulated 10 datasets, each corresponding to one of the 10 experimental datasets (Figure 2-4). These datasets were simulated by using the recovered pulses of the 10 experimental datasets shown in Figure 2-2 as well as the estimated model parameters for each of the corresponding datasets as shown in Table 2.2. Then, we added zero mean Gaussian noise with standard deviations shown in Table 2.3 to the simulated data. These 10 simulated datasets were sampled every 10 minutes.

Table 2.3 shows the estimated model parameters, number of pulses, the square of the multiple correlation coefficient (R2 ), the percentage error in estimating the model parameters, and the standard deviation of zero mean Gaussian noise used in simulating the 10 datasets. These simulations were performed by adding various values of zero mean Gaussian noise with standard deviations o-,, ranging from 0.29 to 1.44.

50 P.0.Iwip 2 Wall is

7b"eM

8It It 1 PM pow4 P mops! 4 20j

I:J 2

to 12 1s1 i is n n 0 4 88 1* 12 11 $2 2 I Pilmu-M

-5 a I

It 12 14 is, is 29 22 4 2 I+s t Is 1 211

,20

Thu wm4

Is' III

2, 4 6 S-s It14 It. I to It 12 4 Is IS Ws 32 %Me To" mak

Pm 20J

Figure 2-2: Estimated Deconvolution of the Experimental Twenty-Four- Hour Cortisol Levels in 10 Women. Each panel shows the measured 24-hour cortisol time series (red stars), the estimated cortisol levels (black curve), the estimated pulse timing and amplitudes (blue vertical lines with dots) for one of the participants. The estimated model parameters are given in Table 2.2.

51 .. 4.0 aL61 g 126...... 1......

S 1 1 2 Sodud H@W616

1 IS 3 P6 2614 12

112

A 9 1 2 3 aia~fdW~auman.2 ai:16 4. 26 2O6 1" 115 1"

.3 4 a1 2 3

...... -......

10 N 4 26 26 126 M 1104 40 1so 1NI 10

j~a 4 . * 1 2 .4

I0 1 2 3 m-i

La5 N6 40 N6 26 126 1AN14

t mmi2-4 1. 1- 42 -10a1 2 0

Figure 2-3: White Gaussian Structure in the Model Residuals of 10 Women. In each panel, (i) the top sub-panel displays the autocorrelation function of the model residuals in one of the 10 participants; the graph shows that the model captures the dynamics and that residuals axe white; (ii) the bottom sub-panel displays the quantile- quantile plot of the model residuals for that participant; the graph shows that the residuals are Gaussian.

52 11WPM P--Udpm 2 VA", I I

a n 12 34 6 19 44 16 It It. 22 24 711ft

-4 0 ix a Is 211 22 24 U it 20 22 At Tuft

VINdw Vjd,-

I Is I

4 0 S .19 12 14 16 0. 26 TMN Th"

PoOkWOM

I I, I ii

4 is ix 14 a 20 2. 4 6 a 12 10 Is 22 34 raw FAW* Thm

9 PON10"16 vans I I ,

4 12 14 is 2.2 24 2 4 9 0 10 34 Tko WON$ Tb"

Figure 2-4: Simulated Twenty-Four-Hour Cortisol Levels with Measurement Errors Corresponding to Datasets from 10 Women. Each panel displays the simulated serum cortisol levels based on pulse patterns in Figure 2-2 and estimated model parameters 01 and 02 in Table 2.2 in one of the 10 participants, assuming a zero mean Gaussian measurement error with standard deviation o,, in Table 2.3. In all simulations the initial conditions are xi(0) = 0, x 2 (0) equals the initial cortisol level of the corresponding participant, and the cortisol levels are recorded every 10 minutes.

53 Table 2.3: The Estimated Model Parameters and the Squares of the Mul- tiple Correlation Coefficients (R2 ) for the Fits to the Simulated Cortisol Time Series.

2 Participant 91(min~') 2(min~') N R .() 0,_ __ _(%) 102-02%) 1 0.0628 0.0068 16 0.99 0.38 15.02% 1.49% 2 0.0569 0.0056 15 0.97 0.75 25.33% 1.75% 3 0.0747 0.0078 15 0.98 0.67 18.89% 4.88% 4 0.0918 0.0071 16 0.94 1.44 26.44% 16.39% 5 0.0492 0.0123 18 0.99 0.52 15.90% 0.82% 6 0.0490 0.0122 19 0.99 0.29 32.51% 28.42% 7 0.0770 0.0098 16 0.97 0.98 3.63% 8.41% 8 0.0375 0.0094 18 0.99 0.33 2.74% 3.30% 9 0.0365 0.0091 16 0.99 0.35 1.11% 1.11% 10 0.0654 0.0076 19 0.99 0.31 24.30% 4.11% Median 0.0599 0.0084 16 0.99 0.45 17.39% 3.70%

The parameters 91 and 92 are, respectively, the estimated infusion rate of cortisol into the circulation from the adrenal glands and the estimated clearance rate of cortisol by the liver. N is the estimated nu nber of hormone pulses and R2 is the square of the multiple correlation coefficieni b. a, is the standard deviation of the zero mean Gaussian noise added to each simulated data point. For each participant, blood samples were assayed in duplicate, and the corresponding standard deviation of noise was obtained. The parameters 01 and 02 are, respectively, the infusion rate of cortisol into the circulation from the adrenal glands and the clearance rate of cortisol by the liver used in simulating each dataset. The values of 01 and 02 are given in Table 2.2.

The level of noise was based on the signal-to-noise ratio for data collection for each of the participants. Errors in estimating 01 and 02 range from small values (1.11% and 0.82%) to high values (32.51% and 28.42%), respectively. The maximum error in finding the number of pulses is two. There is zero error in finding the number of pulses for participant 7 where the maximum error in the detected support is 14 minutes. The overall performance of the estimation for the simulated dataset is best for the data that corresponds to participant 7 with o-, = 0.98, and 3.63% and 8.41% error in estimating 01 and 02, respectively. These estimates were obtained by 10 random initializations, and considering that the optimization problem solved for this estimation is non-convex, there are multiple local minima and the estimation can be improved by using more initializations. Since the noise added to the simulated data

54 is comparable in amplitude to small pulses of cortisol, the pulsatile patterns of the simulated cortisol time series differ from their corresponding experimental time series.

This explains the higher error in the estimated model parameters 01 and 02, and the error in the estimated timing and amplitudes of pulses. This choice of noise is based on the immunoassay error for each experimental time series, so that the simulations are based on multiple metrics of the experimental data. The algorithm performs better for lower levels of noise that do not significantly affect the pulsatile patterns of the simulated time series.

Table 2.4: The Error in Estimated Pulses for the Simulated Cortisol Time Series.

Participant Me (min) N. Nd 1 4 2 0 2 14 2 0 3 6 1 0 4 26 1 0 5 10 1 1 6 22 1 0 7 14 0 0 8 22 0 2 9 13 2 2 10 22 1 0

Me is the maximum error in the detected timing of pulses. N. and Nd are, respectively, the number of pulses that are not detected and the number of extra pulses that are detected for each simulated dataset.

Figure 2-5 shows the actual sparse input, the recovered input, the simulated cortisol data, and the estimated cortisol levels. Table 2.4 shows the maximum error in the detected timing of pulses, the number of small pulses that are not detected, and the number of extra pulses that are detected for each simulated dataset. The recovered and the actual inputs are in good agreement for a variety of noise levels when detecting significant pulses; however, a few of the small pulses cannot be detected for some cases or in some cases noise is captured as a small pulse. The maximum error in detecting the support is 26 minutes.

55 2.6 Discussion

Understanding the cortisol secretion process and modeling the underlying system is a challenging problem for several reasons. (1) Due to the simultaneous release and clearance of hormones and the unknown timing and amplitudes of the secretory events, identifying the pulsatile input to the system and the infusion and clearance rates is challenging. (2) Due to data collection difficulties and cost, the sampling interval is usually relatively large (10-60 minutes) compared to the expected inter-pulse intervals as well as secretion and clearance rates of cortisol. This low resolution of the data makes identifying the delays in the system and potential consecutive pulses that occur over one sampling interval challenging. (3) Cortisol secretion differs in sleep and wake states and at different circadian times; therefore, the model parameters might be time-varying. (4) There is inter-individual variation, even among healthy individuals. (5) The properties of noise in the system are not known.

In this chapter, we modeled secretory events that result in cortisol time series, and proposed a coordinate descent approach to estimate the model parameters and recover the sparse time-varying secretory input. Considering the sparsity of the input, we recovered the impulses using compressed sensing. While a range for the sparsity of the hormone pulses is known, the exact number of pulses varies from one individual to another and is unknown. To recover the accurate number of hormone pulses, the regularized problem should be solved such that there is a balance between capturing the sparsity and the residual error. We used generalized cross-validation for choosing the regularization parameter and finding the number of pulses for each individual. The algorithm described in this chapter provides a general framework that can also be implemented on other hormones. For the case of cortisol data, the high R2 values

56 30 3W 25 1 25

IIs

Thp4 p-lip-u Wi. 30 LE $OF* -1

i1 II ~21

* 24

16 12 14 is Is ft. 22 31 I Thwu no I ii I a.gWi 21

Is 12 14 I is a. f2 Is'

I- Is I II [email protected] Ado.

Is -2

o i a lit I II.- C 2 a Ise

11 hi jti II'

10121411183222 Thu -

Figure 2-5: Estimated Deconvolution of Simulated Twenty-Four-Hour Cor- tisol Levels with Different Measurement Errors Corresponding to Datasets from 10 Women. Each panel shows the simulated 24-hour cortisol time series (blue stars), the estimated cortisol levels (black curve), the simulated pulse timing and amplitudes (blue vertical lines with dots) and the estimated pulse timing and amplitudes (red vertical lines with empty circles) for one of the simulated datasets that each correspond to a participant. The estimated parameters are given in Table 2.3. 57 (found to be greater than 0.92 for all 10 participants) suggest that our proposed algorithm can successfully uncover physiologically plausible hormone pulse information underlying cortisol secretion. There are variations in the timing and amplitudes of cortisol secretory events. The amplitude variations throughout the day occur as a result of the circadian rhythm underlying cortisol release; the variations in the timing of impulses reflect the ultradian rhythm underlying cortisol release. Our algorithm makes it possible to capture the circadian and ultradian features of hormone pulses as well as the parameters underlying the first-order kinetics of cortisol release. Fur- thermore, our algorithm recovered the impulse train input from simulated data with various noise levels and detected the significant pulses; however, depending on the dataset, it misses one or two of the insignificant pulses or captures noise as one or two small pluses. Our approach can be applied to GH, thyroid hormone, and gonadal hormones in a similar fashion. Correlation analysis of the residuals suggests that the model captures the dynamics, and residuals are Gaussian and white.

Many data analysis methods for modeling hormone pulsatility either assume the timing of the impulses belongs to a certain class of stochastic processes or use pulse detection algorithms [20, 46]. These procedures work well when the pulses are readily identifiable and are more challenging when the pulses are more difficult to discern by visual inspection as in the case of cortisol. Johnson et al. [20] and Vidal et al. [46], respectively, analyze LH data using a birth-death process in an MCMC algorithm and a pulse detection algorithm. While our algorithm also can be applied to LH, we analyzed cortisol data, in which the timing of the hormone pulses is not as clearly defined as in LH data. In summary, our proposed algorithm works well even when the pulses are not easily identifiable while still being applicable to cases in which pulses are identifiable by visual inspection. Furthermore, our algorithm does not require assumptions about the inter-arrival times of the pulses, and timings of pulses can be recovered for different classes of distributions of inter-arrival times.

58 Although our proposed algorithm runs on average in less than half an hour, it can be accelerated. In the data that we analyzed, the likelihood of two pulses occurring during small intervals is low because the experiment was conduced when the participants were under relatively low stress, were fed regularly, had low constant activity levels, and the ambient temperature was held constant. We have not tested our algorithm on data from settings in which the likelihood of two impulses occurring in short intervals is high due to external factors such as stress. Moreover, in our analysis, we make the assumption that the pulses occur at any minute; hence, the detected pulses could have happened within a minute before or after the pulse was detected. Using an approach similar to the one proposed here, with an appropriate number of data points, one could bin the data assuming the pulses occur at any second, or even millisecond and detect the pulses with a higher resolution.

59 60 Chapter 3

Quantifying Pituitary-Adrenal Dynamics and Deconvolution of Concurrent Cortisol and Adrenocorticotropic Hormone Data by Compressed Sensing

3.1 Introduction

ACTH and cortisol axe steroid hormones that influence multiple areas of mammalian physiology including metabolism, inflammation and stress [4]. Neural interactions in the hypothalamus result in the release of hormone-releasing hormones (e.g. CRH),

Chapter adopted from Faghih R.T., Dahleh, M.A., Adler G.K., Klerman E.B., and Brown E.N., Quantifying Pituitary-Adrenal Dynamics and Deconvolution of Concurrent Cortisol and Adrenocor- ticotropic Hormone Data by Compressed Sensing, in preparation, 2014.

61 which in turn induce the release of hormones from the pituitary (e.g. ACTH); pituitary hormones induce secretion of hormones from the target glands (e.g. cortisol). These hormones implement regulatory functions in the body and also have a feedback effect on the release of hormone-releasing hormones and pituitary hormones [22]. ACTH and cortisol vary over a 24-hour period as a result of the ultradian modulation of the timing and circadian modulation of the amplitudes of the ACTH and cortisol secretory pulses [4]. In the 1960s, endocrinologists realized that some hormones are not secreted in a continuous manner, but are secreted in pulsatile episodes [47] and cleared exponentially [20]. In order to understand the physiology and psy- chophysiology, effects of drugs, and other interventions, there is a need for quantifying pulsatile episodes of hormone release. It is currently unethical to measure the hormone-releasing hormones of the hypothalamus in human participants, and therefore it is crucial to infer these pulsatile episodes using serum measurements of only a subset of the hormones within these hierarchical hormone systems. A question of interest is whether elevated cortisol levels observed in some medical conditions are caused by changes in the pulsatile episodes or by the increased sensitivity of the adrenal glands to ACTH [52]. One method for investigating whether an endocrine disorder has been initiated by the hypothalamus, the pituitary, or the gland, is to construct a model based on these interactions and develop an algorithm that can find the pulsatile episodes of hormone release in the hypothalamus and in the pituitary as well as the model parameters corresponding to each step of the cascade using data from individuals with and without the disorder. It is therefore advantageous to have a model and an estimation algorithm that can investigate the role of the amplitude and frequency of the pulses as well as the sensitivity of the adrenal glands to the ACTH level. Moreover, a desired treatment could use an optimal dosage (amount and timing) by employing a model that predicts the dose-response. This requires a comprehensive model that includes the pulsatile secretion of hormones and the feedback effect in hormone release, so that hormone secretion and hormone concentration in the blood can be estimated.

62 As a first step in understanding endocrine systems quantitatively, we investigate concurrent release of ACTH and cortisol in the HPA axis. Various models of the HPA axis and cortisol secretion have been proposed; for instance see [4], [9], [8], [16],[12], and [6]. These models describe cortisol synthesis in the adrenal glands based on the first-order kinetics of cortisol synthesis. Current mathematical models for concurrent ACTH and cortisol measurements include [32], [28], [42], [33], and [26]. Peters et al. modeled ACTH and cortisol levels as a function of exogenous CRH [32]. They intravenously injected CRH to the participants, collected ACTH and cortisol levels until 4 hours after injection, and estimated the model parameters given the injected CRH [32]. Another data-driven model of concurrent ACTH and cortisol levels was proposed by Lonnebo et al. where they assumed a surge-based nonlinear model with one morning surge and one afternoon surge [28]. Van Cauter proposed a method for recovering episodic hormone fluctuations [42], used it to analyze the 24-hour proffle of concurrent ACTH and cortisol data and recovered the secretory events for each of the two hormone profiles [33], [26]. Then, by analyzing the timing of the detected

ACTH and cortisol pulse peaks and the respective durations those pulses overlapped, [33] and [26] determined which detected ACTH and cortisol pulses were concomitant. Some data analysis methods for recovering hormone secretory events are developed for single hormone time series (e.g. only cortisol) and are mostly based on pulse detection algorithms [46] or assume that the timing of the secretory events belong to a certain class of stochastic processes [20]. In the previous chapter, we showed that hormone secretory events can be recovered using compressed sensing. Compressed sensing allows for reconstruction of sparse signals (i.e., signals in which only a small number of coefficients are large and most coefficients are zero or close to zero) using fewer measurements than required by the Shannon/Nyquist sampling theorem [3].

Our goal in this study is to quantify the secretory events and the feedback control mechanism that underlies the HPA axis. To tackle this problem, we develop a model that calculates the first-order kinetics underlying the HPA axis and the infusion and

63 clearance coefficients given datasets that include concurrent measurements of ACTH and cortisol serum levels. We use a multi-rate state space representation of the system and by using a coordinate descent approach, we recover the model parameters and the secretory events. Based on the physiology, between 15 to 22 secretory events are expected for the ACTH-cortisol system over 24 hours [4], [43]. Considering that these secretory events are sparse, we employ compressed sensing techniques to recover the secretory events. The exact number of secretory events for each participant is unknown; using generalized cross-validation, we find the number of pulses such that there is a balance between capturing the noise and the sparsity. Since cortisol, growth hormone, thyroid hormone, estrogen, and testosterone are synthesized and secreted using a similar feedback control mechanism, the proposed framework could potentially be applied to all these endocrine hormones.

3.2 Experiment

To test our model, we used serum ACTH and cortisol measurements collected simultaneously; experimental data were from an inpatient study of 10 healthy women. A detailed description of the experiment is in [23] and clinical characteristics of the participants are in the previous chapter. Blood was collected via an indwelling intra- venous catheter every 10 minutes for 24 hours, was assayed for ACTH and cortisol in duplicate, and the immunoassay error for each time series was obtained. The 24 hours began with 8 hours of scheduled sleep followed by 16 hours of wake. The 16 hours of wake during this portion of the experimental protocol included constant routine conditions designed to minimize the effects of stress, posture changes, eating, and ambulatory temperature on the participants. Therefore, this dataset can be used to quantify ACTH and cortisol variations as a function of the time of the day as controlled by the circadian and the ultradian patterns. This project has been reviewed and approved by the Brigham and Women's Hospital IRB. During the review of this project, the IRB specifically considered (i) the risks and anticipated benefits, if any,

64 to subjects; (ii) the selection of subjects; (iii) the procedures for securing and documenting informed consent; (iv) the safety of subjects; and (v) the privacy of subjects and confidentiality of the data.

3.3 Modeling Formulation

We build our model based on the stochastic differential equation model of diurnal cortisol patterns in [4]. This model is based on first-order kinetics for cortisol synthesis in the adrenal glands, cortisol infusion to the serum, and cortisol clearance by the liver. It uses a doubly stochastic pulsatile input with gamma distributed interarrival times and a Gaussian circadian amplitude [4]. This input marks pulsatile cortisol synthesis in the adrenal glands.

We represent the secretory events of the anterior pituitary using an impulse train. This impulse train marks the timing and amplitude of the secretory events that result in ACTH synthesis in the anterior pituitary. We assume that there are between 15 to 22 secretory events that control the 24-hour serum cortisol level, based on [4], [43].

We also include the known cortisol negative feedback effect on ACTH secretion [4], [16], [24], [9], [8] in our model. Equations (3.1)-(3.3) model the HPA axis and cortisol and ACTH release:

dx1 (t) Pituitary) (3.1) dt = 1x1(t) - 02 x3(t) + u(t) (Anterior

dX (t) = 2 03 x 1 (t) - 04 x2 (t) (Adrenal Glands) (3.2) dt dX 3(t) = 04 x2 (t) - 95X 3(t) (Serum) (3.3)

where x1 is the serum ACTH concentration, x 2 is the cortisol concentration in the adrenal glands, x3 is the serum cortisol concentration, and 03 is the ACTH gain.

65 01 and 02 represent the infusion rate of ACTH from the anterior pituitary to the blood and the cortisol negative feedback gain, respectively. 94 and 95 represent the coefficients corresponding to infusion of cortisol into the circulation from the adrenal glands and clearance of cortisol by the liver, respectively. u(t) is an abstraction of the secretory events in the anterior pituitary that result in ACTH release and consequent cortisol release: u(t) = -ri) (3.4)

where qj denotes the amplitude of a secretory event initiated at time ri, and m denotes

the number of the secretory events. Our goal is to estimate the model parameters (9A

for j = 1, 2, ... , 5), the number of the secretory events (m), and the amplitudes (qi for

i = 1, 2, ... , m) and timing (r for i = 1, 2, ..., m) of the secretory events using serum

ACTH (x1) and cortisol (x3 ) levels collected in 10-minute intervals.

We start by putting the control feedback model of ACTH and cortisol secretion (equations (3.1)-(3.3)) in a state space form, and writing the discrete analog of the system.

We obtain a system of equations with unknowns 93 (for j = 1, 2, ... , 5), m, qj and ri (for i = 1, 2, ... , m). - 01 0 -02 1

B0 , C= 0 Let x(t) = Xi X2 X3], A =:3 -04 ] 0 04 -05 101 serum y(t) = [yi(t) y2(t)], v(t) = [v(t) v2(t)] where yi and y2 are the observed

levels of ACTH and cortisol, respectively, and vi(t) and v2(t) represent ACTH and cortisol assay noise, respectively. Hence, the ACTH-cortisol model can be represented in the state space form as:

x = Ax(t) + Bu(t)

y(t) = Cx(t)+v(t)

Assuming that the input and the states are constant over one-minute intervals (T=1), by letting

66 form:

x[k +1] =qx[k]+ru[k]

y[k] = Cx[k] + v[k]

Considering that the serum ACTH and cortisol levels are observed every 10 minutes, 8 by letting Ad = 410, Bd = [9r 0 r ... r], ud[k] = [u[10k] u[10k + 1] ...u[10k + 9] vd[k] = v[10k] and z[k] = x[10k], we can represent the multi-rate system as:

z[k + 1] = Adz[k] + Bdud[k]

y[k] = Cz[k-+ vd[k] where Ad and Bd are functions of 9 = [01 ... 05 . Then, using the state transition matrix, and considering that the system is causal and N data points are available, we can represent the system as:

y[k] = F[k]zo + D[k]u + vd[k] where F[k] = CA', zo = z[0], D[k] = C [A 1Bd Adk 2 Bd - Bd , and [ N-kJ

U1ONx3 = [ud[0] Ud[11 .. ' ud4k - 1] ... ud[N - 1]]. u represents the entire input over the duration of the study, sampled every minute. Considering de novo cortisol synthesis, no cortisol is stored in the adrenal glands [4], so we can assume that the initial condition of the cortisol level of the adrenal glands is zero. Assuming that the initial observed ACTH and cortisol levels are the initial conditions, we can 1 let zo = z[O] = [y1[0] 0 y2[0Ij . Then, let y = y[ ] y[2] ...y[N] 2NxI, where y represents all the data points. Moreover, let Fe = [F[O] F[11 F[N - 1] 2Nx3

Do = D[] D[1] .- D[N - 1]J 2Nx1ON, and vy = [Vd[1] Vd[2] - vd[N] . Hence, we can represent this system as:

67 y = Fozo + Deu + vy where F and Do are functions of 0 and the sparse vector u. Let YA, FA, DA, and vA correspond to the odd rows of y, F, Do, and vy, respectively (i.e. quantities corresponding to ACTH secretion), and let yc, Fee, D9 0 , and vc correspond to the even rows of y, FO, Do, and vy, respectively (i.e. quantities corresponding to cortisol secretion); then, the system can equivalently be represented as:

YA = FeAzo + DeAu + VA

yc Fe zo + Dou + vc. where YA represents serum ACTH levels, yc represents serum cortisol levels collected at 10-minute intervals, and zo is a vector of the initial conditions of the serum ACTH concentration, the adrenal glands' cortisol concentration, and the serum cortisol concentration. u represents the entire input over 24 hours. Elements of u take nonzero values qj at times r1 for i = 1, 2, ... , m when there is a secretory event, and are zero otherwise. FA, FOA, DOA, and De are functions of 9, for j = 1, 2,..., 5.

3.4 Model Estimation

Between 15 to 22 secretory events are expected for the ACTH-cortisol system over 24 hours [4], [43], so we assume the minimum number of secretory events umin = 15 and the maximum number of secretory events um.. = 22. Considering that a pulse can occur at any minute, we assume u contains umin to Umax nonzero elements out of 1440 possibilities, and all these nonzero elements are nonnegative (Umin < liUvlO < Umin, u > 0). Since hormone gains, infusion, and clearance coefficients cannot be negative, we assume that 9 > 0. Furthermore, we follow [4] and assume that the infusion

68 coefficient of cortisol from the adrenal glands to the circulation is at least four times the clearance coefficient of cortisol by the liver (405 04 ). We can formulate this problem as an optimization problem:

mm -TIIYA-Fe12 + 11~IYC i~

2 CDeU11 min y- - Deu| + ||y - FCZo - 2 (3.5) s.t. Umin 5 |ul0 Um.

U ;> 0

SO < q

0 0 0 -1 4 - -1 0 0 0 0 0 0 0 -1 0 0 0 where S =,and q =0 - A and qB represent the stan- 0 0 -1 0 0 0 0 0 0 -1 0 0 0 0 0 0 -1 - dard deviation of the ACTH and cortisol measurement errors, respectively. In this optimization problem, solving for u is a combinatorial problem, which is generally NP-hard, and is solved using greedy algorithms and i,-optimization algorithms. The greedy algorithms include Matching Pursuit (MP), Othogonal MP, Iterative Hard

Thresholding, Hard Thresholding Pursuit, Gradient Descent with Sparsification, and

Compressive Sampling Matching Pursuit [18]. In the t,-optimization algorithms, the to-norm is approximated by an tp-optimization problem where 0 < p < 2 [18]. i,-optimization algorithms are more accurate than greedy algorithms, but computationally more expensive [18]. It is possible to cast the above optimization problem as:

U>1 2o C S

69 where the i,-norm is an approximation to the i0-norm (0 < p <; 2) and A is chosen such that the sparsity of u is between umi to umx. Then, using a coordinate descent approach, this optimization problem can be solved iteratively using the following steps until convergence is achieved:

1. U(+1 = argminJ\(9(l), u) (3.7) u>O

2. 0(1+1) = argminJA(9, u( 1+) (3.8) SO~q

The optimization problem in (3.7) can be solved using the FOCUSS algorithm [14]. The FOCUSS algorithm is based on the iteratively reweighted least squares algorithm, and enforces a certain degree of sparsity [53]. The sparsity is determined by A (the sparsity of u increases with A), and A balances between sparsity and the residual error. We use an extension of the FOCUSS algorithm called GCV-FOCUSS+ [10]. The GCV-FOCUSS+ algorithm is based on FOCUSS+ [30] that solves for nonnegative u such that u has a certain maximum sparsity n (i.e., n = 22 for the HPA axis), and uses the generalized cross-validation technique [13] for estimating the regularization parameter. In particular, GCV-FOCUSS+ is closely related to a version of the FO- CUSS algorithm by Zdunek et al. [53], which uses the GCV technique for updating the regularization parameter A. Choosing a A value that balances between the noise and sparsity is important in detecting the sparsity level. If A is too small, overfitting can occur and noise can be detected as signal; on the other hand, if A is too large, it leads to underfitting the data, and as a result, the signal will not be constructed com- pletely. The FOCUSS+ algorithm and the GCV-FOCUSS+ algorithm are described in more detail in the previous chapter.

An important factor in estimating the model parameters and the input is the initialization of u and 9: this should be done systematically. It is possible to obtain a

70 good initial estimate for the timing of the pulses that result in cortisol secretion by first deconvolving the cortisol data using the model and the algorithm in the previous chapter. This algorithm detects the significant pulses that result in cortisol release. Some peaks might not appear to be significant given only cortisol measurements and the corresponding pulses might not be detected; however, when analyzing the ACTH and cortisol data together these pulses might now appear significant. Hence, we first find the peaks in the cortisol data, and then break the data into peak-to-peak segments, and then for the segments that no pulse is detected, we allow for the support (i.e., the timing of pulses) to occur any time over that segment. We use the findpeaks function in MATLAB R2011b to compare each cortisol data point to its neighboring values, and then, by finding the data points that take values higher than both of their neighbors, we detect the peaks in the cortisol data. This process gives a good initial condition for the possible support of u; the following summarizes the algorithm for obtaining good initial conditions for 0 and u:

1. Using the deconvolution algorithm in the previous chapter, obtain an estimate of the timings and amplitudes of pulses that result in cortisol secretion (u')

2. Find the peaks in the cortisol data, and break the data into peak-to-peak segments

3. For each segment, if there is no pulse detected in u., set u0 for that segment equal to a vector of all 1s; otherwise, set uO for that segment equal to those entries of uc that correspond to that segment

4. Initialize 0 randomly

5. Set equal to #-1; using FOCUSS+, solve for 6' by initializing the optimization problem in (3.7) at u0

6. Set 5 equal to 0k; using the Levenberg-Marquardt method, solve for 0k by

initializing the optimization problem in (3.8) at 9 k1

7. Iterate between steps 5 and 6 for a selected number of iterations

71 8. Set 90 and 0 equal to the 0 and i values that minimize JA(0, u) in (3.6).

This initialization algorithm is based on the algorithm we reported in the previous chapter; the model parameters and the input that minimize JA(0, u) in (3.6), denoted by 90 and 0', respectively, are good initializers for the main estimation algorithm. The following is the algorithm that we propose for estimation of concurrent measurements of cortisol and ACTH:

1. Initialize the algorithm at 60 and 6' using the initialization algorithm

2. Set 0 equal to 6k1; using GCV-FOCUSS+, solve for 0k by initializing the optimization in (3.7) at 6k- 1

3. Set 6 equal to 0k, and using the Levenberg-Marquardt method, solve for 6' by initializing the optimization in (3.8) at $k-I

4. Iterate between steps (2) and (3) until convergence

5. Repeat steps (1)-(4) for various initializations

6. Set the estimated model parameters 9 and L equal to the values that minimize Jx(9, u) in (3.6).

The optimization problem in (3.6) is non-convex and there are multiple local minima; the proposed algorithm selects the set of model parameters and the input that give the best goodness of fit. We have implemented the algorithm under the assumption that hormone pulses occur at integer minutes. The serum ACTH measurements are by 4 orders of magnitude smaller than serum cortisol measurements, and this leads to issues for numerical analysis due to the difference in the order of magnitude, the much smaller ACTH appears as noise. To handle numerical issues in running the estimation algorithm, we scaled up the ACTH measurements by a factor of 4 orders of magnitude. Then, we scaled down the estimated 02 by 4 orders of magnitude and

72 scaled up the estimated 93 by 4 orders of magnitude to compensate for scaling up ACTH in the estimation. This step balances the units in the system of equations that describe the HPA axis. For evaluating the R2 , we estimated the ACTH and cortisol levels using scaled 9 values (reported in Table 3.1), and ACTH and cortisol initial conditions with units M. Also, the impulse input has units 2 so that all equations are balanced when evaluating the R2 . In implementing the GCV-FOCUSS+ algorithm, we solve for u by letting p = 0.5. Data analysis, estimation, and simulations were performed in MATLAB R2011b.

3.5 Results

Figure 3-1 shows the experimental 24-hour ACTH and cortisol levels in 10 women. As expected, ACTH and cortisol levels are time-varying functions with regular periodic patterns, and the 24-hour serum ACTH and cortisol profiles consist of episodic hormone release with varying magnitudes in a regular diurnal pattern. For most participants, the ACTH and cortisol levels are low at the beginning of the scheduled sleep. Then, around the wake time, the ACTH and cortisol levels rapidly increase, followed by a gradual decrease throughout the day. Figure 3-2 shows the model- predicted ACTH and cortisol estimates, and the estimated amplitudes and timing of hormone pulses for each participant. The timing and amplitudes of the detected hormone pulses vary over a 24-hour period across participants. The amplitudes of the recovered pulses have a circadian rhythm; for most participants, there are fewer recovered pulses at the beginning of the scheduled sleep, and there is a large pulse towards the end of the sleep period or beginning of the wake period. There are multiple small and medium sized pulses during the wake period.

The number of recovered pulses for all participants is within their corresponding phys-

73 Purdpant 1 Partcipand 2

0 2 4 C 1 10 12 14 16 13 21 22 24

. 2 4 0 510 2 14 16 13 21 22 24 Irii iat Th. Pum) PwUcIlpn4

I 2 40

* 2 4 6 1 1. 12 14 if I. 20 22 24 ~20 2 4 0 10 12 14 If 1S 21 22 24 Partepipat2 .30 U IM20 *1 is 3 * 2 4 6 It 12 14 I6 10 26 22 24 * 2 4 S 0 IS 12 14 16 1S 2. 22 24 Th" PM*M4

* ;2 4 1 12 14 Ift IS 20 22 24 . 2 4 6 1 10 12 14 If 18 2. 22 24 ParnsIpunt 2 I * 2 4 S 1 it 12 14 16 If 20 22 24 I U3 I I

* 2 4 6 0 15 12 14 1f 1t 21 22 24 Th"

P-1. I * 2 4 6 1 It 12 14 16 13 20 n2 24

It 651 12 14 16 16 24 22 24 I 8

It 2 222 * 2 4 6 S 10 12 14 16 13 20 22 24 mm9w 4

I I ii * 2 4 0 9 If 12 14 1f 15 21 22 24 MMhOU

Figure 3-1: Twenty-Four-Hour Serum ACTH and Cortisol Levels in 10 Women. In each panel, (i) the top sub-panel displays the ACTH level and (ii) the bottom sub-panel displays the cortisol level in the corresponding participant. The ACTH and cortisol levels are recorded every 10 minutes. The shaded gray area corresponds to sleep period and the white area corresponds to wake period.

74 R ME Emig gal 9;a-- I

I U

now

T1 T

W.FA .Jzj:js

in,

J T'r

an. M U..

I ~ I'.

Figure 3-2: Estimated Deconvolution of the Experimental Twenty-Four- Hour Concurrent ACTH and Cortisol Levels in 10 Women. In each panel, (i) the top sub-panel shows the measured 24-hour ACTH time series (red stars), and the estimated ACTH levels (black curve), (ii) the middle sub-panel shows the measured 24-hour cortisol time series (red stars), and the estimated cortisol levels (black curve), (iii) the bottom sub-panel shows the estimated pulse timing and amplitudes (blue vertical lines with dots) using concurrent measurements of ACTH and cortisol for the corresponding participant. The shaded gray area corresponds to sleep period and the white area corresponds to wake period. The estimated model parameters are given in Table 3.1. 75 Table 3.1: The Estimated Model Parameters for the Fits to the Experimen- tal ACTH and Cortisol Time Series

Participant 02(m.--) I0(m.-~1) 94(mi-1) 0m-1) 1 0.0043 4.2x~io7 561 0.5044 0.0510 2 0.0046 2.4x 10- 7 530 0.6513 0.0549 3 0.0054 2.5x 10- 7 997 0.9162 0.1055 4 0.0082 1.6x 10- 7 335 0.9788 0.0506 5 0.0054 2.8x10~ 7 804 0.9959 0.1092 6 0.0062 1.9 X10~ 7 792 0.9995 0.0914 7 0.0060 1.3x10- 7 1161 0.4494 0.1122 8 0.0070 1.4x 10-7 382 0.7089 0.0573 9 0.0045 1.1x10- 7 205 0.1953 0.0488 10 0.0035 2.6x 10- 7 548 0.8185 0.0919 Median 0.0054 2.2x 10-7 554 0.7637 0.0743

The parameter 01 is the estimated infusion rate of ACTH from the anterior pituitary into the circulation; 02 is the estimated cortisol negative feedback gain; 03 is the estimated ACTH gain; 04 is the estimated coefficient corresponding to infusion of cortisol into the circulation from the adrenal glands and 05 is the estimated coefficient corresponding to clearance of cortisol by the liver.

iologically plausible ranges [4, 43] and varies among participants. The squares of the multiple correlation coefficients (R2 ) are between 0.82 and 0.94 for cortisol time series and between 0.46 and 0.79 for ACTH time series using concurrent measurements of ACTH and cortisol time series (Table 3.2). Figure 3-3 shows the autocorrelation function and the quantile-quantile plots of the ACTH model residual errors for the 10 participants suggesting that the model captures the ACTH dynamics, and that the ACTH residual errors have a Gaussian structure and are white. Figure 3-4 shows the autocorrelation function and the quantile-quantile plots of the cortisol model residual errors for the 10 participants suggesting that the model captures the cortisol dynamics, and that the cortisol residual errors have a Gaussian structure and are white.

The number of pulses obtained using concurrent measurements of ACTH and cortisol range from 15 to 18 pulses; the number of pulses obtained only from cortisol measurements varies from 15 to 20 pulses. Figure 3-5 shows the estimated secretory events of

76 PiUdOia 1 Pa 1bN* 2 U U I I 41 20 4 6 36U164 120 144 I Lee I

.2 2 .1 a 1 2 4 4 I Nasp 0.44 6A& 24 44 6 124 144 P-U1h" 2

-2 w4 um 04 2

* 20 44 N4 N 1N 126 144 S 4 9 44 a4 4 1"0 124 144

%40

LLa Uldd m 0U

j 4%.- 1 2 1 *04 6 64 1 1249 1"4

I LIt

*0 2 6 4 6 N 164 1284 1U 3 Lao ...... 2. 2

puaddpmiS j21

.2 -2 6 1 2 2 2 4 4 64 1 6 12 144ti gUmid Memfhi m j L" ++

Figure 3-3: White Gaussian Structure in the Model Residual Errors of ACTH Levels in 10 Women. In each panel, (i) the top sub-panel displays the autocorrelation function of the ACTH model residual errors in one of the 10 participants; the graph shows that the model captures the dynamics and that ACTH residual errors are white; (ii) the bottom sub-panel displays the quantile-quantile plot of the ACTH model residual errors for that participant; the graph shows that the ACTH residual errors are Gaussian. 77 PaitClpat 2

N2 40 80 N0 100 110 140 00 40 U0 IN10 120 40 Lag Lay

.1 0 2 2

2 1 2 2 00 40 SO 0 I00 120 140

S..

;3.2 Anna 0u4 2. 211 240 N0 0 100 120 3 Lag Stiw oIO~~ I UIP

-2~ 2 2 Sbmdwd NMni QaUMOM

0.57

N0 N0 IN10 120 140 40 00 20 0 1IN 120 146

Lsa 3 a uudwmsnu Gu.0Uea 10

I o 6I N4 0 29 1

as Mdrdf oudgaia 45' 29 4 0 U0 U0 16 120 140 Los

I. +on+~0 +

.2 1 2 2

Figure 3-4: White Gaussian Structure in the Model Residual Errors of Cor- tisol Levels in 10 Women. In each panel, (i) the top sub-panel displays the autocorrelation function of the cortisol model residual errors in one of the 10 participants; the graph shows that the model captures the dynamics and that cortisol residual errors are white; (ii) the bottom sub-panel displays the quantile-quantile plot of the cortisol model residual errors for that participant; the graph shows that the cortisol residuals are Gaussian. 78 Table 3.2: The Estimated Number of Pulses and the Squares of the Multiple Correlation Coefficients (R2 ) for the Fits to the Experimental ACTH and Cortisol Time Series

participant IM INI R$ 71_RA 1 18 18 0.94 0.74 2 17 17 0.88 0.65 3 16 16 0.94 0.66 4 17 17 0.88 0.73 5 18 15 0.89 0.52 6 20 15 0.92 0.58 7 16 17 0.91 0.51 8 16 16 0.89 0.79 9 16 17 0.83 0.67 10 20 17 0.82 0.46 Median 17 17 0.89 0.655

M is the estimated number of cortisol pulses using only measurements of cortisol levels, and N is the estimated number of ACTH pulses using concurrent measurements of ACTH and cortisol levels. R$ and RA are the square of the multiple correlation coefficient for cortisol and ACTH time series, respectively.

the anterior pituitary recovered using concurrent ACTH and cortisol measurements as well as the estimated secretory events of the adrenal glands recovered using the cortisol measurements. The timing of most of the significant pulses recovered from concurrent measurements of ACTH and cortisol are in agreement with the timing of most significant pulses recovered only from cortisol measurements for most participants; however, some of the less significant pulses are not in agreement in terms of timing or have not been recovered. The mismatch between the detected ACTH secretory events and cortisol secretory events could be due to the noise that exists in the HPA axis during the secretion process as well as measurement noise or some nonlinearities in the HPA axis. These results suggest that using only cortisol serum measurements one can find the timing of most of the significant secretory events in the HPA axis (i.e. the timing of pulses that are crucial in both cortisol and ACTH profiles). By including ACTH serum measurements, either some of the pulses that were detected using only cortisol data are not detected anymore or some new pulses are captured that were not detected previously. Also, there can be small differences

79 in the timing of the detected pulses depending on whether cortisol data was used alone or both ACTH and cortisol data were used. The amplitude of the recovered

ACTH secretory events are by 4 orders of magnitude smaller than those of cortisol, which is expected as the amplitudes of the ACTH data are by 4 orders of magnitude smaller than those of the cortisol data.

94 and 95 are estimated coefficients corresponding to the infusion and clearance of cortisol, respectively. These coefficients are functions of the underlying infusion and clearance rates of cortisol, respectively, which we reported in the previous chapter. Gains on ACTH are larger than the cortisol infusion and clearance coefficients by 4 orders of magnitude and negative feedback gains on cortisol are smaller by 4 orders of magnitude than the infusion rate of ACTH by the anterior pituitary. These differences in the orders of magnitudes are expected as ACTH serum levels are smaller than cortisol serum levels by 4 orders of magnitude. On average, after scaling up the ACTH measurements by a factor of 4 orders of magnitude and referring to the standard deviation of the scaled-up ACTH measurements as 0 A, we have UA ; V2c.

3.6 Discussion

Using a data-driven linear dynamical system model for ACTH and cortisol release that includes the effect of secretory events over 24 hours, we can describe and quantify multiple important components of the HPA axis. In chapter 2, we modeled cortisol secretion using first-order kinetics underlying cortisol secretion and recovered the secretory events that lead to cortisol release and the corresponding model parameters. A similar approach could also be used for deconvolution of ACTH data without including the negative feedback effect of cortisol. However, a secretory event is crucial for regulation of the HPA axis if it is significant for both ACTH and cortisol

80 S. I *1 "I I

n n 14-0

I I~f I I is I

I "I "I I "1 .1 'I

I I lamp-"

I I I t ti I

I 41~ OLLi I.. I I I

Figure 3-5: Comparison of Estimated Pulse Timing and Amplitudes Using the Experimental Twenty-Four-Hour Cortisol Levels Only with Estimates from Concurrent ACTH and Cortisol Levels in 10 Women. Each panel shows the estimated pulse timing and amplitudes (blue vertical lines with dots) using concurrent measurements of ACTH and cortisol, and the estimated pulse timing and amplitudes (red vertical lines with dots) using only cortisol measurements for the corresponding participant. The shaded gray area corresponds to sleep period and the white area corresponds to wake period. 81 release. Having a model that includes interactions between ACTH and cortisol allows for recovering the secretory events that are significant in regulation of the HPA axis as a whole. The pulses that are detected for one hormone, but not for both hormones, might be capturing noise in the HPA axis or the data.

Modeling the interactions between ACTH and cortisol by including concurrent data from both hormones allows for understanding the input/output relationship between ACTH and cortisol. Currently, stimulation tests are used to diagnose hormonal disorders that are caused by a problem in the anterior pituitary or the adrenal glands. Using data from a large population of healthy human subjects, a model and a deconvolution algorithm that includes the ACTH gain and cortisol negative feedback like the model proposed in this chapter, we could find a range for model parameters for the healthy population. This could then be used to recognize where some hormonal disorders of the HPA axis are originating from for specific patients. For example, if a patient has elevated cortisol levels, the model can distinguish whether the negative feedback effect of cortisol on ACTH is too low, or the gain on ACTH is too high, or the clearance rate of cortisol by the liver is too low. The algorithm provided in our study is a novel way of recovering the feedback gain, ACTH gain, and ACTH secretory events simultaneously. Furthermore, having a model for cortisol synthesis makes it possible to analyze the HPA axis as a whole and can potentially lead to a systematic approach for treating disorders related to cortisol and also for achieving normal cortisol secretion in an optimal manner.

Modeling the HPA axis using concurrent serum ACTH and cortisol measurements is a challenging problem. (1) Due to data collection costs, the sampling interval of 10 minutes is relatively large compared to the inter-pulse intervals, secretion and clearance rates, and the delay in the HPA axis. This large sampling interval makes it challenging to identify the potential consecutive pulses that occur over one sampling interval as well as the potential delays over one sampling interval. (2) Since

82 ACTH decays faster than cortisol, due to the low resolution of the data, the ACTH response to a small pulse might have already decayed out while the cortisol response might be observed in the collected data, and distinguishing between such a small pulse and noise is challenging. (3) The properties of noise in the HPA axis are not known. (4) The HPA axis is highly coupled with the circadian rhythm, which varies over 24 hours; therefore, the model parameters might be time-varying. (5) There is inter-individual variation, even among healthy individuals. (6) Some of the significant periods of ACTH data are different from some of the significant periods of cortisol data; changes in the significant periods in the two datasets might be due to some nonlinearities in the input-output relation of ACTH and cortisol, or caused by the noise in the data or the HPA axis. (7) Sometimes there is a pulsatile secretory event in ACTH data without a response in cortisol data or there is a secretory event in cortisol data without ACTH stimulation, which makes modeling ACTH and cortisol interactions using a simple linear model challenging.

The described approach to estimation of ACTH and cortisol measurements is a general approach that can similarly be applied to concurrent measurements of other hormones. For cortisol data, the high R2 values (above 0.82 for all 10 participants) suggest that our proposed algorithm can successfully reveal physiologically plausible information regarding cortisol release in the HPA axis. Considering that the half-life of ACTH is less than that of cortisol and a sampling interval of 10 minutes can not capture all characteristics of the ACTH profile (e.g. the peak values), the current ACTH data is not as reliable as the current cortisol data; this may explain why the R2 values obtained for the ACTH data were lower than the R2 values found for the cortisol data. The variations in the hormone data also might be due to undersampling (when the sampling frequency is low), and experimental error that can be caused by assay variability and pipetting [15]. While we used a deterministic approach for deconvolution of cortisol levels, it is possible to cast the parameter estimation problem as a Bayesian estimation problem by considering a Gaussian distribution for the f 2 -

83 norm and a Laplace distribution for the ei-norm in the cost function in (3.6), and obtain confidence intervals for the model parameters and the pulses underlying cortisol secretion.

In this chapter, we used a multi-rate state space approach in estimation of the HPA axis. We formulated the problem in a way that allows for concurrent deconvolution of two hormone profiles, while considering one hormone (ACTH) as the input resulting in the release of the other hormone (cortisol). The proposed multi-rate state space framework can be applied to other hierarchical endocrine hormone systems (e.g. thyroid-stimulating hormone and thyroid hormone). We implicitly fit the ACTH data, reduced the effect of the high noise in the data, and compensated for the difference between the significant periods of ACTH and cortisol data (i.e. some of the significant frequencies of the ACTH data were not present in the cortisol data or vice versa) and the cases in which there is a pulse of ACTH data without a cortisol response or vice versa. The proposed approach is also beneficial for analyzing concurrent measurements when one of the observations is very noisy; using the proposed approach, one can fit the noisy data implicitly and reduce the effect of noise when performing estimation.

84 85 Chapter 4

An Optimization Formulation for Characterization of Pulsatile Cortisol Secretion

4.1 Introduction

Except for prohormones, all other hormones that have been well-investigated appear to be released in pulses [40]; for example, cortisol, gonadal steroids, parathormone, and insulin are released in a pulsatile manner [44]. Pulsatility is a physiological way of increasing hormone concentrations rapidly and sending distinct signaling information to target cells [44]. Ultradian pulsatile hormone secretion allows for encoding information via both amplitude and frequency modulation and is a way of frequency encoding [48], [25]. Pulsatile signaling permits target receptor recovery, rapid changes in hormone concentration, and greater control, and is also more efficient than con-

Chapter adopted from Faghih R.T., Dahleh, M.A., and Brown E.N., An Optimization Formu- lation for Characterization of Pulsatile Cortisol Secretion, in preparation, 2014.

86 tinuous signaling [48]. The mechanism underlying the generation of hormone pulses and why this method of signaling is chosen by the body over continuous signaling is not known. Since the transcriptional program prompted by hormone pulses is con- siderably different from constant hormone treatment [40], it is crucial to understand the physiology underlying pulsatile hormone release. In particular, it is essential to understand the physiology behind pulsatile cortisol secretion for various reasons. (I) Cortisol oscillations have crucial effects on target cell gene expression and glucocorticoids receptor function [29], [50]. (II) Some psychiatric and metabolic diseases are linked to changes in cortisol pulsatility [49]. (III) When the same amount of corticosterone is used by constant infusion rather than a pulsatile infusion, it results in a noticeably reduced ACTH response to stress [25]. In this chapter, we investigate pulsatile release of cortisol and propose a novel mathematical formulation that char- acterizes pulsatile cortisol secretion.

Cortisol is released from the adrenal glands in pulses in response to pulsatile release of ACTH. CRH induces the release of ACTH. In return, cortisol has a negative feedback effect on ACTH and CRH release at the pituitary and hypothalamic levels. The timing and amplitudes of cortisol pulses vary throughout the day where the amplitude variations are due to the circadian rhythm underlying cortisol release with periods of 12 and 24 hours [9], and the variations in the timing of cortisol pulses result from the ultradian rhythm underlying cortisol release. Between 15 to 22 secretory pulses of cortisol are expected over 24 hours [4], [43].

Based on the interactions in the HPA axis, it was hypothesized that pulsatile release of CRH from the hypothalamus results in pulsatile release of cortisol. Walker et al. suggest that a sub-hypothalamic pituitary-adrenal system results in the pulsatile ultradian pattern underlying cortisol release [50]. This is because inducing constant CRH levels results in a pulsatile cortisol profile [50] while constant ACTH levels do not result in pulsatile cortisol secretion [39]. Spiga et al. suppressed the activity of the

87 HPA axis by oral methylprednisolone and infused both constant amounts and pulses of ACTH to test the hypothesis that pulsatile ACTH release is necessary for pulsatile cortisol secretion [39]. While pulsatile ACTH resulted in pulsatile cortisol secretion, constant infusion of same amounts of ACTH did not activate cortisol secretion [39]. Moreover, studies on sheep in which the hypothalamus has been disconnected from the pituitary suggest that pusatile CRH secretion is not necessary for the ultradian rhythm in cortisol secretion and pulsatile cortisol secretion is still maintained [49]. Hence, the dynamics in the anterior pituitary control pulsatile cortisol release. Since pulsatile cortisol release seems to be more efficient than continuous signaling, it might be the case that the anterior pituitary is solving an optimal control problem.

We postulate that there is a controller in the anterior pituitary that controls the pulsatile secretion of cortisol and the ultradian rhythm of the pulses via the negative feedback effect of cortisol on the anterior pituitary. Hence, by considering the known physiology of the HPA axis, we shall formulate an optimization problem that achieves impulse control. In optimal control theory, impulse control is a special case of bang- bang control, in which an action leads to instantaneous changes in the states of the system [38]. Impulse control occurs when there is not an upper bound on the control variable and an infinite control is exerted on a state variable in order to cause a finite jump [38]. Minimizing an to-norm cost function can achieve impulse control and we use a reweighed e 1-norm formulation as a relaxation to the to-norm to solve the proposed optimization formulation. Moreover, we consider the first-order dynamics underlying cortisol synthesis and the circadian amplitude constraints on the cortisol levels when formulating the optimization problem.

88 4.2 Methods

We propose a physiologically plausible optimization problem for cortisol secretion by making the following assumptions: (1) Cortisol levels can be described by first-order kinetics for cortisol synthesis in the adrenal glands, cortisol infusion to the blood, and cortisol clearance by the liver described in [4, 10, 8, 9]. (2) There is a time-varying cortisol demand (h(t)) that should be satisfied throughout the day, which is a function of the circadian rhythm. (3) There is a time-varying upper bound on the cortisol level (q(t)) that is a function of the upper bound on the cortisol level that the body can produce or a holding cost so that the cortisol level would not be much above the demand. (4) Control that results in cortisol secretion (u(t)) is nonnegative. (5) The body is minimizing the number of resources (control) throughout the day. Hence, we postulate that there is a controller in the anterior pituitary that controls cortisol secretion via the following optimization formulation:

minm ||U110u(4.1)

s.t. u(t) > 0 dxj(t) = -Axi(t) + u(t)

d"(t) = Axl(t) - yx 2 (t)

h(t) < x 2 (t) q(t)

where x, is the cortisol concentration in the adrenal glands and x2 is the blood cortisol concentration. A and y, respectively, represent the infusion rate of cortisol from the adrenal glands into the blood and the clearance rate of cortisol by the liver. Considering the known physiology of de novo cortisol synthesis (i.e., no cortisol is stored in the adrenal glands) [4], we assume that the initial condition of the cortisol level in the adrenal glands is zero (x1 (0) = 0) [4]. Assuming that the input and the

89 states are constant over one-minute intervals, and yo is the initial condition of the blood cortisol concentration, blood cortisol levels at every minute over N minutes

can be represented in discrete form by y = Yl Y2 YN]' where Yk is the blood cortisol level at time k and y can be represented as:

y = Fyo + Gu (4.2)

where F = [fi f2 - fN] ' k = e-"k, G [gi g2 - gN]

\ and urepresents the control k = - e-Ak) . N-k

over N minutes. Then by letting h = [h, h2 - hN] where hk is the cortisol de-

mand at an integer minute k and q = [qi q2 - qN where qk is the upper bound at the integer minute k. Hence, we solve the discrete analog of the formulation in (4.1):

min ||ullo ""X (4.3)

s.t. U > 0

x = Fyo + Gu h

to problems are generally NP-hard, and instead an e1 -norm relaxation of such problems can be solved. In solving 4-norm problems, there is a dependence on the am-

plitude of the coefficients over which the e1 -norm is minimized, and there is more penalty on larger coefficients than on smaller ones. However, it is possible to strate- gically construct a reweighted 4-norm such that nonzero coefficients are penalized in a way that the cost further resembles the to-norm. By putting large weights on small entries, the solution concentrates on entries with small weights, nonzero entries are discouraged in the recovered signal, and a cost function that is more similar to

90 an to-norm cost function can be solved [5]. To find such weights for 4-norm cost function, Candes et al. [5] have proposed an iterative algorithm for enhancing the sparsity using reweighted 4i minimization, which solves minl ullo. This algorithm is U based on Fazel's "log-det heuristic" algorithm for minimizing the number of nonzero entries of a vector [11] and the convergence of this log-det heuristic algorithm has been studied in [27]. Hence, we use the algorithm by Candes et al. [5] such that the constraints in the optimization problem in (4.3) are satisfied:

1. Initialize the diagonal matrix W(') with entries w ") = 1, i = 1,...,n on the diagonal and zeros elsewhere

2. Solve u) =arg minIW(V)uIJi subject to the constraints in (4.3) U 3. Update the weights e+1) 11 Z -uleIE7 i 7 1, ..,

4. Iterate till f reaches a certain number of iterations. Otherwise, increment i and go to step 2.

The idea is that by solving u(+) =arg minn iteratively, the algorithm at- tempts to solve for a local minimum of a concave penalty function that is more similar to the to-norm [5]. E is used to ensure that weights on the recovered zero entries will not be set to oo at the next step, which would prevent us from obtaining estimates at the next step. e should be slightly larger than the expected nonzero amplitudes of the signal that is to be recovered, and a value of at least 0.001 is recommended [5].

This algorithm does not always find the global minimum and as E -+ 0, the likelihood of stagnating at an undesirable local minimum increases [5]. For e values closer to zero, the iterative reweighted e1 -norm algorithm stagnates at an undesirable local minimum [5].

We study the optimization problem in (4.1) via four examples. We first investigate the case that the optimization formulation in (4.1) is selecting the control such that

91 the state is bounded between constant lower and upper bounds to illustrate the idea that the formulation in (4.1) can achieve impulse control. Then, we investigate cases in which the upper and lower bounds have harmonic profiles with a circadian rhythm. Using the iterative algorithm for enhancing the sparsity by reweighted f, minimization [5], we solve the optimization problem in (4.1) over a time period r and update the solution after a time period Z and repeat this process for a 24-hour period. A, -y, e, T, and lower and upper bounds are given in Tables 4.1, 4.2, and 4.3. Since empirically the algorithm converges in 10 iterations for the formulation in this study, we use f = 10 when running the algorithm. Numerical analysis was performed in MATLAB R2011b.

Table 4.1: Model Parameters for Examples of Optimization Problem (4.1)

Example I A(min-1) I7(min-') ( )"I r (min) 1 0.0585 0.0122 0.01 360 2 0.0585 0.0122 0.0055 360 3 0.0585 0.0122 0.0075 360 4 0.1248 0.0061 0.0075 360

The parameters A and -y are, respectively, the infusion rate of cortisol into the circulation from the adrenal glands and the clearance rate of cortisol by the liver, and were both obtained from chapter 2. The parameter E provides stability for the iterative algorithm for enhancing the sparsity by reweighted 4i minimization [5], and r is the period over which we solve the iterative algorithm.

Table 4.2: Upper Bounds for Examples of Optimization Problem (4.1)

Example q(t)(") 1 14 2 5.3782 + 0.3939sin( 6 ) - 3.5550cos(-M) - 0.5492sin(2) + 1.0148cos( ) 3 8.6051+ 3.0306sin( 4 ) - 5.0931cos(40) - 1.8151sin() - 1.6570cos() 4 8.6051 + 3.0306sin( 1 .t) - 5.0931cos( ) - 1.8151sin($) - 1.6570cos(2) q(t) is the upper bound on the cortisol level.

92 4.3 Results

Example 1 - Assuming that the upper and lower bounds are constant, the optimal solution is achieved when the initial condition starts at the upper bound; then, when the state decays to the lower bound, an impulse causes a jump in the state which brings it back to the upper bound, and then again the state decays to the lower bound and the same jump to the upper bound again occurs, and the same process keeps repeating. Figure 4-1 shows that solving the optimization problem (4.1) for constant upper and lower bounds using the parameters given for Example 1 in Table 4.1 and the upper and lower bounds provided in Tables 4.2 and 4.3, respectively, results in impulse control. There are 12 constant impulses obtained over a 24-hour period, which occur periodically. This example is just a simple toy problem illus- trating that the optimization formulation in (4.1) can achieve impulse control and pulsatile cortisol release using a low energy input. This example does not have any physiological implications- for cortisol secretion as it does not include upper and lower bounds that have a circadian rhythm observed in cortisol levels.

Example 2 - In healthy humans, cortisol levels have regular periodic time-varying patterns that consist of episodic release of secretory events with varying timings and amplitudes in a regular diurnal pattern. Figure 4-2 shows that solving the optimization problem (4.1) for two-harmonic bounds with a circadian rhythm, using the parameters given for Example 2 in Table 4.1 and the upper and lower bounds pro-

Table 4.3: Lower Bounds for Examples of Optimization Problem (4.1)

Example h(t)(f) 1 6 2 3.2478 - 0.7813sin(-2-) - 2.8144cos(') - 0.2927sin(k) + 1.3063cos(2-t) 3 5.5065 + 1.5544sin( ) - 4.3112cos( ) - 1.6355sin( ) 0.9565cos( 4 5.5065 + 1.5544sin( 2) - 4.3112cos(4-) - 1.6355sin($) - 0.9565cos(L72) h(t) is the lower bounds on the cortisol level.

93 15

10 -

Z5N

0' 0 2 4 6 8 10 12 14 16 18 20 22 24 Time (hour) p 5 -- 0 - 5-

0 0 2 4 6 8 10 12 14 16 18 20 22 24 ime (hour)

Figure 4-1: Cortisol Levels and Control Obtained Using Example 1 (i) The top panel displays the optimal cortisol profile (black curve), constant upper bound (red curve), and constant lower bound (blue curve). (ii) The bottom panel displays the optimal control. The optimization problem obtained 12 impulses over 24 hours as the optimal control (the timing of the control was discretized into 1440 points; the obtained control takes 12 nonzero values, i.e. impulses, while it is zero everywhere else). The optimization problem was solved using the parameters given in Example 1 in Table 4.1 and the upper and lower bounds provided in Tables 4.2 and 4.3, respectively.

94 vided in Tables 4.2 and 4.3, respectively, the obtained control is impulse control. There are 16 impulses over a 24-hour period with time-varying circadian amplitudes and ultradian timings; the obtained control is within the physiologically plausible range of 15 to 22 pulses [41, [43]. The impulses are more frequent during the day and have higher amplitudes during the day than in night time. Obtained cortisol levels are low at night. Then, around 6 AM, cortisol levels increase, reaching higher values between 10 AM to 12 PM, followed by a gradual decrease throughout the day reaching low values at night. The obtained control and state are optimal; the state starts at the upper bound and decays to the lower bound at which point an impulse causes a jump in the system that results in increasing the state, and the state reaches the upper bound. Then, the state decays again to the time-varying lower bound and this process repeats. This example illustrates that the optimization formulation in (4.1) can achieve impulse control and pulsatile cortisol release, using a low energy input, and generate secretory events and cortisol levels that have physiologically plausible profiles similar to those observed in healthy human data.

Example 3 - In this example, we consider different lower and upper bounds com-

pared to Example 2 while keeping A and -y to values used in Example 2. Figure 4-3 shows that solving the optimization problem (4.1) for two-harmonic bounds with a circadian rhythm, using the parameters given for Example 3 in Table 4.1 and the upper and lower bounds provided in Tables 4.2 and 4.3, respectively, the obtained control is impulse control. 16 impulses are obtained over 24 hours which is within the physiological range of 15 to 22; these impulses have time-varying circadian amplitudes and ultradian timings. The impulses have higher amplitudes and are more frequent between 4 AM to 12 PM. The obtained cortisol levels are low at night. Then, the cortisol levels increase, reaching higher values between 7 AM to 11 AM, followed by a gradual decrease throughout the day, reaching low values at night. This example illustrates that the optimization formulation in (4.1) can achieve impulse control and pulsatile cortisol release using a low energy input, and generates secretory events and

95 121

10-

0 4-

0 2 4 6 8 10 12 14 16 18 20 22 24 Time (hour) 10 -

5 -

0 0 2 4 6 8 10 12 14 16 18 20 22 24 Time (hour)

Figure 4-2: Cortisol Levels and Control Obtained Using Example 2 (i) The top panel displays the optimal cortisol profile (black curve), two-harmonic upper bound (red curve), and two-harmonic lower bound (blue curve). (ii) The bottom panel displays the optimal control. The optimization problem obtained 16 impulses over 24 hours as the optimal control (the timing of the control was discretized into 1440 points; the obtained control takes 16 nonzero values, i.e. impulses, while it is zero everywhere else). The optimization problem was solved using the parameters given in Example 2 in Table 4.1 and the upper and lower bounds provided in Tables 4.2 and 4.3, respectively.

96 15-

110-

L0 2 4 6 8 10 12 14 16 18 20 22 24 Time (hour) C 5 5- ~I1 o0[ 0 ____5-IIITTYYTT U0 0 2 4 6 8 10 12 14 16 18 20 22 24 Time (hour)

Figure 4-3: Cortisol Levels and Control Obtained Using Example 3 (i) The top panel displays the obtained cortisol profile (black curve), two-harmonic upper bound (red curve), and two-harmonic lower bound (blue curve). (ii) The bottom panel displays the obtained control. The optimization problem obtained 16 impulses over 24 hours as the control (the timing of the control was discretized into 1440 points; the obtained control takes 16 nonzero values, i.e. impulses, while it is zero everywhere else). The optimization problem was solved using the parameters given in Example 3 in Table 4.1 and the upper and lower bounds provided in Tables 4.2 and 4.3, respectively.

cortisol levels that have physiologically plausible profiles similar to those observed in healthy human data. The control and state obtained in the first 20 hours are optimal; however, the control and the state obtained for the last 4 hours are suboptimal as the algorithm used for solving the optimization problem (4.1) can stagnate at a local minimum depending on the choice of E. However, still a low energy control is recovered that keeps the cortisol levels within the desired bounds.

Example 4 - In this example, we keep the lower and upper bounds the same as the values we used in Example 3 while using values for A and -y that result in higher

97 infusion of cortisol and lower clearance of cortisol compared to Example 3. Figure 4-4 shows that solving the optimization problem (4.1) using the parameters given for Example 4 in Table 4.1 and the upper and lower bounds provided in Tables 4.2 and 4.3, respectively, the obtained control is impulse control. Twelve impulses are obtained over 24 hours where the impulses have lower amplitudes and are less frequent compared to the impulses obtained in Example 3. The obtained impulses still have time-varying circadian amplitudes and ultradian timings. The number of pulses has decreased compared to Example 3 which was expected as cortisol is cleared faster in this example. While the number of these pulses are not within the physiological range reported for healthy subjects, the obtained cortisol levels are still within the desired range. Cortisol levels are low at night, then increase, reaching higher values between 6 AM to 10 AM, followed by a gradual decrease throughout the day, reaching low values at night. The peak values of cortisol levels change and on average in this example the cortisol levels have lower values, and this might illustrate a case of cortisol deficiency. Also, in this example, the optimization formulation in (4.1) results in impulse control and pulsatile cortisol release using a low energy input. The control and state obtained in the first 19 hours are optimal; however, the control and the state obtained for the last 5 hours are suboptimal as the algorithm used for solving the optimization problem (4.1) can stagnate at a local minimum depending on the choice of c. However, still a low energy control is recovered that keeps the cortisol levels within the desired bounds.

4.4 Discussion

It is well-known that cortisol is released in pulses, and based on our results it appears that this method of relaying information might be an optimal approach as opposed to continuous signaling. In this work, we formalized this concept by proposing an optimization formulation for a physiologically pluasible controller in the anterior pituitary

98 15-

10 -

5 -

0 2 4 6 8 10 12 14 16 18 20 22 24 C Time (hour) 15 - 6 10 - 5 -

0 0 2 4 6 8 10 12 14 16 18 20 22 24 Time (hour)

Figure 4-4: Cortisol Levels and Control Obtained Using Example 4 (i) The top panel displays the obtained cortisol profile (black curve), two-harmonic upper bound (red curve), and two-harmonic lower bound (blue curve). (ii) The bottom panel displays the obtained control. The optimization problem obtained 12 impulses over 24 hours as the control (the timing of the control was discretized into 1440 points; the obtained control takes 12 nonzero values, i.e. impulses, while it is zero everywhere else). The optimization problem was solved using the parameters given in Example 4 in Table 4.1 and the upper and lower bounds provided in Tables 4.2 and 4.3, respectively.

99 that achieves impulse control as the optimal solution. In the proposed formulation, we assumed that there is a time-varying upper bound on the cortisol levels in the plasma. Also, we assumed that the cortisol levels in the plasma should be above a time-varying circadian threshold to achieve normal regulation of the HPA axis. We assumed that the lower bound and upper bound on the cortisol levels are two-harmonic functions with periods of 12 and 24 hours that are controlled by the circadian rhythm. How- ever, the upper bound and the lower bound for cortisol secretion could have multiple harmonics, and this assumption is only considering the most significant periods in cortisol release. Moreover, we considered the first-order dynamics underlying cortisol secretion. We have shown that the proposed optimization formulation yields impulse control as its optimal solution. The number, timing, and amplitude of the recovered secretory events in the proposed optimization problem are physiologically plausible. Moreover, the obtained cortisol profile is in agreement with the circadian rhythm observed in healthy human data. As pointed out, the iterative algorithm for enhancing the sparsity by reweighted i minimization [5] does not always find the global minimum and might stagnate at an undesirable local minimum; we em- ployed this algorithm to solve examples of optimization problems formulated in (4.1) to show that the formulation in (4.1) achieves impulse control as observed in cortisol levels. However, problem (4.1) can be solved using other methods as well, and for arbitrary choices of E and r the algorithm for enhancing the sparsity by reweighted f, minimization [5] might stagnate at local minima and not achieve the optimal solution.

While we proposed a simple optimization formulation that can achieve impulse control, it is possible to obtain impulse control using more complex formulations by either assuming that the system is a switched system with different rates or assuming that the nature of the system is impulsive and there is no continuous control. We assumed that the infusion and clearance rates are constant; however, the system can be a switched system with different infusion and clearance rates. Abrupt changes in the infusion and clearance rates could also result in impulse control. For example, if the

100 infusion rate of cortisol from the adrenal glands starts from a constant level at wake and decreases abruptly to a new constant level, a very large level of cortisol should be produced in a short time so that the desired cortisol level can still be achieved. There could be multiple abrupt changes in the infusion rate throughout the day, and there might be an infusion rate reset to a high level at the beginning of sleep. Another example that could possibly result in impulse control is when the clearance starts at a constant level, and increases abruptly to a new constant level; then, a very large level of cortisol should be produced in a short time so that the desired cortisol level can still be achieved. There could be multiple such abrupt changes in the clearance rate throughout the day, and the clearance rate might be reset to a low level at the beginning of sleep. Another scenario could be that both the infusion and the clearance rates could be starting from a constant level and change abruptly to different levels periodically. In that case, the overall effect is that cortisol gets cleared faster or cortisol gets infused to the plasma more slowly, and at such moments a very large cortisol level should be released for a short period of time to maintain the desired cortisol level. Such situations could possibly achieve impulse control as long as there is not an upper bound on the control variable; a mathematical example of a model with a time-varying rate that achieves impulse control is given in [38], and the maximum principle is used to find the optimality conditions for this problem. Moreover, it is possible that pulsatile inputs arise from the nature of the system, and the hormone system might be designed such that the input to the system can only be impulsive where the timing of the impulses are functions of the states and are not activated until a reseting condition is satisfied. A mathematical example of such a model is given in [51] where the cost function minimizes the energy in the input and the state, and calculus of variations is used to find the optimality conditions. Also, another possibility is that the body is solving a weighted f, cost function where different costs are associated with the control at different times of the day (e.g. the weights obtained at convergence when using the reweighted algorithm).

101 In this study, for modeling cortisol secretion, we proposed a physiologically plausible optimization formulation for a controller in the anterior pituitary. A similar approach can be used to study other endocrine hormones that are released in pulses. For example, the proposed optimization formulation can be tailored to include the constraints underlying thyroid hormone secretion or gonadal hormone secretion or growth hormone secretion in order to study the pulsatile release of those hormones. The transcriptional program stimulated by hormone pulses is very different from constant hormone treatment and some disorders are linked to hormone pulsatility. Hence, understanding the underlying nature of the pulsatile release of these hormones via mathematical formalization can be beneficial to understanding the pathological neuroendocrine states and treating some hormonal disorders.

102 103 Chapter 5

Conclusion and Future Work

5.1 Conclusion

Understanding how one endocrine hormone is released adds insight to understanding the rest of the endocrine hormones. This thesis investigated cortisol secretion and the interactions between ACTH and cortisol release, and gave a mathematical characterization of cortisol secretion to better understand the role of pulses in cortisol release. Understanding the control mechanism underlying cortisol release is a key step in potentially obtaining an optimal dosage policy for treating cortisol disorders and reducing the side effects of the drugs used for treating such disorders.

5.1.1 Deconvolution of Serum Cortisol Levels by Compressed

Sensing

Using only the observed serum cortisol levels, we modeled cortisol secretion from the adrenal glands using a second-order linear differential equation and represented the sparse cortisol pulses using an impulse train. Using our algorithm and the physio-

104 logically plausible assumption that the number of pulses is between 15 to 22 pulses over 24 hours, we successfully deconvolved both simulated datasets and actual 24-hr serum cortisol datasets sampled every 10 minutes from 10 healthy women. Assuming a one-minute resolution for the secretory events, we obtained physiologically plausible timings and amplitudes for each cortisol secretory event with R2 above 0.92.

5.1.2 Quantifying Pituitary-Adrenal Dynamics and Decon- volution of Concurrent Cortisol and Adrenocorticotropic Hormone Data by Compressed Sensing

For serum ACTH and cortisol levels sampled at 10-minute intervals from 10 healthy women, we modeled ACTH and cortisol secretion using a system of linear differential equations, and represented the sparse ACTH pulses using an impulse train, and recovered the model parameters and physiologically plausible timing and amplitudes of ACTH pulses. We solved this deconvolution problem under the physiologically plausible assumption that there are between 15 to 22 ACTH pulses over 24 hours. Our results have R2 between 0.82 and 0.94 for cortisol and between 0.46 and 0.79 for ACTH time series.

5.1.3 An Optimization Formulation for Characterization of Pulsatile Cortisol Secretion

While it is well-known that some hormones such as cortisol are released in pulses, it is not known why it is optimal for the body to release hormones in pulses as opposed to releasing them in a continuous manner. We hypothesize that there is a controller in the anterior pituitary that leads to pulsatile release of cortisol, and propose a

105 mathematical formulation for such controller, which leads to impulse control as opposed to continuous control. This novel approach results in impulse control where the impulses and the obtained plasma cortisol levels have a circadian rhythm and an ultradian rhythm that are in agreement with the known physiology of cortisol secretion. Our results suggest the possibility that a controller in the anterior pituitary regulates the pulsatile release of cortisol.

5.2 Future Work

The algorithms and methods used in this thesis can be tailored to study growth hormone, thyroid hormone, and gonadal hormones. By having access to a large dataset, we can use the proposed approach to identify a range for model parameters that correspond to healthy participants and distinguish normal and abnormal physiology. Moreover, using the models proposed in this thesis, in the future it may be possible to design optimal approaches for treating hormonal disorders.

While it is currently unethical to measure the hormone-releasing hormones of the hypothalamus (e.g. CRH) in human participants, it possible to measure some of such hormones in large mammals such as sheep, and by obtaining concurrent CRH, ACTH, and cortisol data, it would be possible to include the hypothalamic interactions in studying the HPA axis as a whole. Moreover, by obtaining simultaneous data from multiple hypothalamic neurons, another future direction would be to mathematically model the coupling among hypothalamic neurons and how these neurons synchronize.

106 107 Bibliography

[1] Attouch H, Bolte J, Redont P, Soubeyran A (2010) Proximal alternating minimization and projection methods for nonconvex problems: An approach based on the kurdyka- lojasiewicz inequality. Mathematics of Operations Research 35: 438457.

[21 Belmaker R.H., Agam G. (2008) Major depressive disorder, New England Journal of Medicine, 358(1):55-68.

[3] Boufounos P, Duarte M, Baraniuk R (2007) Sparse signal reconstruction from noisy compressive measurements using cross validation. In: IEEE Workshop on Statistical Signal Processing. pp. 299-303.

[4] Brown E.N., Meehan P.M., Dempster A.P. (2001) A stochastic differential equation model of diurnal cortisol patterns, American Journal of Physiology - En- docrinology And Metabolism, 280(3): E450-E461.

[5] Candes, E. J., Wakin, M. B., Boyd, S. P. (2008). Enhancing sparsity by reweighted fi minimization. Journal of Fourier analysis and applications, 14(5-6), 877-905.

[6] Conrad M., Hubold C., Fischer B., Peters A. (2009) Modeling the hypothalamus pituitary adrenal system: homeostasis by interacting positive and negative feedback, Journal of Biological Physics, 35(2):149-162.

108 [7] Dallman M.F. and E.F. Yates. (1969) Dynamic Asymmetries in the Corticos- teroid Feedback Path and Distribution-Metabolism-Binding Elements of the Adrenocortical System, Ann. NY Acad. Sci., vol. 156: 696-721.

[8] Faghih R.T. (2010) The FitzHugh-Nagumo model dynamics with an application to the hypothalamic pituitary adrenal axis, S.M. thesis, Massachusetts Institute of Technology.

[9] Faghih R.T., Savla K., Dahleh M.A., Brown E.N. (2011) A feedback control model for cortisol secretion, Annual International Conference of the IEEE Engi- neering in Medicine and Biology Society, 716 -719.

[10] Faghih R.T., Dahleh, M.A., Adler G.K., Klerman E.B., and Brown E.N., (2014) Deconvolution of serum cortisol levels by using compressed sensing, PLOS ONE 9(1): e85204.

[11] Fazel, M. (2002). Matrix rank minimization with applications, PhD thesis, Stan- ford University.

[12] Frank V., Andersen M., Ottesen J.T. (2011) The minimal model of the hypothalamic pituitary adrenal axis, Journal of Mathematical Biology, 63(4):663-690.

[13] Golub, G. H., Heath, M., Wahba, G. (1979) Generalized cross-validation as a method for choosing a good ridge parameter, Technometrics, 21(2), 215-223.

[14] Gorodnitsky, I.F., Rao, B.D. (1997) Sparse signal reconstruction from limited data using FOCUSS: a re-weighted minimum norm algorithm, IEEE Transac- tions on Signal Processing, 45(3):600-616.

[15] Gudmundsson A., Carnes M. (1997) Pulsatile adrenocorticotropic hormone: An overview, Biological Psychiatry, 41(3):342 - 365.

[16] Gupta S, Aslakson E, Gurbaxani B.M., Vernon S.D. (2007) Inclusion of the glucocorticoid receptor in a hypothalamic pituitary adrenal axis model reveals bistability, Theoretical Biology and Medical Modeling, 4:8.

109 [17] Hansen PC (2000) The 1-curve and its use in the numerical treatment of inverse problems. In: Computational Inverse Problems in Electrocardiology, ed. P. Johnston, Advances in Computational Bioengineering. WIT Press, pp. 119142.

[18] He Z., Xie S., Cichocki A. (2012) On the convergence of FOCUSS algorithm for sparse representation, CoRR,abs/1202.5470, http://arxiv.org/abs/1202.5470.

[19] Hopkins R.L., Leinung M.C. (2005), Exogenous Cushing's syndrome and glucocorticoid withdrawal, Endocrinol Metab Clin N Am, Vol. 34:371-384.

[20] Johnson T.D. (2003) Bayesian deconvolution analysis of pulsatile hormone concentration profiles, Biometrics, 59(3):650-660.

[21] Keenan DM, Chattopadhyay S, Veldhuis JD (2005) Composite model of time- varying appearance and disappearance of neurohormone pulse signals in blood. Journal of Theoretical Biology 236: 242255.

[22] Kettyle W.M., Arky R.A. (1998) Endocrine pathophysiology, Philadelphia: Lippincott-Raven.

[23] Klerman E.B. et al. (2001) Circadian rhythms of women with fibromyalgia, Jour- nal of Clinical Endocrinology & Metabolism, 86(3):1034-1039.

[24] Kyrylov V, Severyanova L.A., Vieira A (2005) Modeling Robust Oscillatory Behavior of the Hypothalamic-Pituitary-Adrenal Axis, IEEE Transaction on Biomedical Engineering, Vol. 52, No. 12:1977-1983.

[25] Lightman S.L., Conway-Campbell, B.L. (2010) The crucial role of pulsatile ac-

tivity of the HPA axis for continuous dynamic equilibration, Nat Rev Neurosci, 11:710-718.

[26] Linkowski, P. et al. (1985). The 24-Hour Profile of Adrenocorticotropin and Cortisol in Major Depressive Illness*. The Journal of Clinical Endocrinology & Metabolism, 61(3), 429-438.

110 [27] Lobo, M. S., Fazel, M., Boyd, S. (2007). Portfolio optimization with linear and fixed transaction costs. Annals of Operations Research, 152(1), 341-365.

[28] Lonnebo A, Grahn6n A, Karlsson M.O. (2007) An integrated model for the effect of budesonide on ACTH and cortisol in healthy volunteers, British Journal of Clinical Pharmacology, 64(2):125132.

[29] McMaster, A., Jangani, M., Sommer, P., Han, N., Brass, A., et al. (2011). Ul- tradian cortisol pulsatility encodes a distinct, biologically important signal. PloS one, 6(1), e15766.

[30] Murray J.F. (2005) Visual recognition, inference and coding using learned sparse overcomplete representations, Ph.D. thesis, University of California, San Diego.

[31] Nillni E.A., Aird F., Seidah N.G., Todd R.B., and Koenig J.I. (2001) Prepro- TRH178199 and Two Novel Peptides (pFQ7 and pSE14) Derived from Its Pro- cessing, Which Are Produced in the Paraventricular Nucleus of the Rat Hypotha- lamus, Are Regulated during Suckling, Endocrinology, Vol. 142: 896-906.

[32] Peters A, Conrad M, Hubold C, Schweiger U, Fischer B, Fehm H.L. (2007) The principle of homeostasis in the hypothalamus-pituitary-adrenal system: new insight from positive feedback, Biometrics, 59(3):650-660. American Journal Phys- iology Regulatory, Integrative and Comparative Physiology, 293(1):R83-98.

[33] Refetoff, S. et al. (1985). The Effect of Dexamethasone on the 24-Hour Proffles of Adrenocorticotropin and Cortisol in Cushing's Syndrome*. The Journal of Clinical Endocrinology & Metabolism, 60(3), 527-535.

[34] Rhee S.S., Pearce E.N. (2011) The Endocrine System and the Heart: A Review, Revista Espanola de Cardiologia (English Edition), Vol. 64(3):220-231, ISSN 1885-5857.

[35] Roelfsema F, van den Berg G, Frlich M, Veldhuis JD, van Eijk A, et al. (1993) Sex-dependent alteration in cortisol response to endogenous adrenocorticotropin. Journal of Clinical Endocrinology & Metabolism 77: 23440.

111

1 1 i . [36] Sarabdjitsingh R.A., Joels M., de Kloet E.R. (2012) Glucocorticoid pulsatility and rapid corticosteroid actions in the central stress response, Physiology & amp; Behavior, 106(1):73 - 80.

[37] Savic D. and S. Jelic (2005) A mathematical model of the hypothalamo-pituitary- adrenocortical system and its stability analysis, Chaos Solutions and Fractals, Vol. 26:427-436.

[38] Sethi, S. P. and Thompson, G. (2000). Optimal control theory : applications to management science and economics, Boston : Kluwer Academic Publishers.

[39] Spiga, F., Waite, E. J., Liu, Y., Kershaw, Y. M., Aguilera, G., Lightman, S. L. (2011). ACTH-dependent ultradian rhythm of corticosterone secretion. En- docrinology, 152(4), 1448-1457.

[40] Stavreva, D. A., Wiench, M., John, S., Conway-Campbell, B. L., McKenna, M. A., Pooley, J. R., et al. (2009). Ultradian hormone stimulation induces glucocorticoid receptor-mediated pulses of gene transcription. Nature Cell Biology, 11(9), 1093-1102.

[41] Ten S., New M., and Maclaren N. (2001), Addison's Disease 2001, JCEM, 86: 2909-2922.

[42] Van Cauter, E. (1981). Quantitative methods for the analysis of circadian and episodic hormone fluctuations. In Human Pituitary Hormones (pp. 1-28). Springer Netherlands.

[43] Veldhuis J.D., Iranmanesh A., Lizarralde G, Johnson M.L. (1989) Ampli- tude modulation of a burstlike mode of cortisol secretion subserves the circadian glucocorticoid rhythm. American Journal of Physiology-Endocrinology And Metabolism, 257: E6-E14.

[44] Veldhuis, J. D. (2008). Pulsatile hormone secretion: mechanisms, significance and

evaluation. In Ultradian Rhythms from Molecules to Mind, Springer Netherlands, 229-248.

112 [45] Veldhuis JD, Keenan DM, Pincus SM (2008) Motivations and methods for analyzing pulsatile hormone secretion. Endocrine Reviews 29: 823864.

[46] Vidal A, Zhang Q, Medigue C, Fabre S, Clement F (2012) Dynpeak: An algorithm for pulse detection and frequency analysis in hormonal time series. PLOS ONE 7: e39001.

[47] Vis D.J., wt al. (2010) Endocrine pulse identification using penalized methods and a minimum set of assumptions, American Journal of Physiology - Endocrinol- ogy And Metabolism, 298(2):E146-E155.

[48] Walker, J. J. et al. (2010) Encoding and decoding mechanisms of pulsatile hormone secretion, Journal of Neuroendocrinology, 22(12):1226-1238.

[49] Walker, J. J., Terry, J. R., Lightman, S. L. (2010). Origin of ultradian pulsatility in the hypothalamicpituitaryadrenal axis. Proceedings of the Royal Society B: Biological Sciences, 277(1688), 1627-1633.

[50] Walker, J. J., Spiga, F., Waite, E., Zhao, Z., Kershaw, Y., Terry, J. R., Lightman,

S. L. (2012). The origin of glucocorticoid hormone oscillations. PLoS biology, 10(6), e1001341.

[51] Wang, X., and Balakrishnan, S. N. (2008). Optimal neuro-controller synthesis for variable-time impulse driven systems. In American Control Conference, 2008 (pp. 3817-3822). IEEE.

[52] Young E.A., Carlson N.E., Brown M.B. (2001) Twenty-four-hour ACTH and cortisol pulsatility in depressed women, Neuropsychopharmacology, 25:267-276.

[53] Zdunek R., Cichocki A. (2008) Improved M-FOCUSS algorithm with overlapping blocks for locally smooth sparse signals, IEEE Trans. Signal Process, 56(10):4752- 4761.

113