A GENERALIZED ESTIMATING EQUATIONS APPROACH TO CAPTURE-RECAPTURE CLOSED POPULATION MODELS: METHODS AND APPLICATIONS Md. Abdus Salam Akanda Thesis presented to the University of Évora to obtain the Degree of Doctor in Mathematics Area of Specialization: Statistics SUPERVISOR: Professor Russell Gerardo Alpizar Jara, PhD ÉVORA, JUNE, 2014 INSTITUTE FOR ADVANCED STUDIES AND RESEARCH Abstract Wildlife population parameters, such as capture or detection probabilities, and density or population size, can be estimated from capture-recapture data. These estimates are of particular interest to ecologists and biologists who rely on ac- curate inferences for management and conservation of the population of interest. However, there are many challenges to researchers for making accurate inferences on population parameters. For instance, capture-recapture data can be considered as binary longitudinal observations since repeated measurements are collected on the same individuals across successive points in times, and these observations are often correlated over time. If these correlations are not taken into account when estimating capture probabilities, then parameter estimates will be biased, possibly producing misleading results. Also, an estimator of population size is generally biased under the presence of heterogeneity in capture probabilities. The use of covariates (or auxiliary variables), when available, has been proposed as an alter- native way to cope with the problem of heterogeneous capture probabilities. In this dissertation, we are interested in tackling these two main problems, (i) when capture probabilities are dependent among capture occasions in closed population capture-recapture models, and (ii) when capture probabilities are heterogeneous among individuals. Hence, the capture-recapture literature can be improved, if we could propose an approach to jointly account for these problems. In summary, this dissertation proposes: (i) a generalized estimating equations (GEE) approach to model possible effects in capture-recapture closed population studies due to cor- relation over time and individual heterogeneity; (ii) the corresponding estimating iv equations for each closed population capture-recapture model; (iii) a comprehen- sive analysis on various real capture-recapture data sets using classical, GEE and generalized linear mixed models (GLMM); (iv) an evaluation of the effect of ac- counting for correlation structures on capture-recapture model selection based on the ‘Quasi-likelihood Information Criterion (QIC)’; (v) a comparison of the per- formance of population size estimators using GEE and GLMM approaches in the analysis of capture-recapture data. The performance of these approaches is eval- uated by Monte Carlo (MC) simulation studies resembling real capture-recapture data. The proposed GEE approach provides a useful inference procedure for esti- mating population parameters, particularly when a large proportion of individuals are captured. For a low capture proportion, it is difficult to obtain reliable esti- mates for all approaches, but the GEE approach outperforms the other methods. Simulation results show that quasi-likelihood GEE provide lower standard error than partial likelihood based on generalized linear modelling (GLM) and GLMM approaches. The estimated population sizes vary on the nature of the existing correlation among capture occasions. Keywords: Capture-recapture experiment; Correlation structure; Generalized estimating equations; Generalized linear mixed models; Heterogeneity; Popula- tion size estimation; Quasi-likelihood information criterion. UMA ABORDAGEM DE EQUAC¸ OES˜ DE ESTIMAC¸ AO˜ GENERALIZADAS PARA MODELOS DE CAPTURA- RECAPTURA EM POPULAC¸ AO˜ FECHADAS: METODOS´ E APLICAC¸ OES˜ Resumo Parˆametros populacionais em esp´ecies de vida selvagens, como probabilidades de captura ou detec¸c˜ao, e abundˆancia ou densidade da popula¸c˜ao, podem ser estima- dos a partir de dados de captura-recaptura. Estas estimativas s˜ao de particular interesse para ecologistas e bi´ologos que dependem de inferˆencias precisas para a gest˜ao e conserva¸c˜ao das popula¸c˜oes. No entanto, h´amuitos desafios para os investigadores fazer inferˆencias precisas de parˆametros populacionais. Por ex- emplo, os dados de captura-recaptura podem ser considerados como observa¸c˜oes longitudinais bin´arias uma vez que s˜ao medi¸c˜oes repetidas coletadas nos mesmos indiv´ıduos em pontos sucessivos no tempo, e muitas vezes correlacionadas. Se essas correla¸c˜oes n˜ao s˜ao levadas em conta ao estimar as probabilidades de cap- tura, as estimativas dos parˆametros ser˜ao tendenciosas e possivelmente produzir˜ao resultados enganosos. Tamb´em, um estimador do tamanho de uma popula¸c˜ao ´e geralmente enviesado na presen¸ca de heterogeneidade das probabilidades de cap- tura. A utiliza¸c˜ao de co-vari´aveis (ou vari´aveis auxiliares), quando dispon´ıveis, tem sido proposta como uma forma de lidar com o problema de probabilidades de captura heterog´eneas. Nesta disserta¸c˜ao, estamos interessados em abordar dois problemas principais em mode1os de captura-recapturar para popula¸c˜ao fechadas, (i) quando as probabilidades de captura s˜ao dependentes entre ocasi˜oes de captura, e (ii) quando as probabilidades de captura s˜ao heterog´eneas entre os indiv´ıduos. Assim, a literatura de captura-recaptura pode ser melhorada, se pud´essemos pro- por uma abordagem conjunta para estes problemas. Em resumo, nesta disserta¸c˜ao vi prop˜oe-se: (i) uma abordagem de estima¸c˜ao de equa¸c˜oes generalizadas (GEE) para modelar poss´ıveis efeitos de correla¸c˜ao temporal e heterogeneidade individual nas probabilidades de captura; (ii) as correspondentes equa¸c˜oes de estima¸c˜ao gen- eralizadas para cada modelo de captura-recaptura em popula¸c˜ao fechadas; (iii) uma an´alise sobre v´arios conjuntos de dodas reais de captura-recaptura usando a abordagem cl´assica, GEE e modelos linear generalizados mistos (GLMM); (iv) uma avalia¸c˜ao do efeito das estruturas de correla¸c˜ao na selec¸c˜ao de modelos de captura-recaptura com base no ‘crit´erio de informa¸c˜ao da Quasi-verossimilhan¸ca (QIC)’; (v) uma compara¸c˜ao da performance das estimativas do tamanho da popula¸c˜ao usando GEE e GLMM. O desempenho destas abordagens ´eavaliado usando simula¸c˜oes Monte Carlo (MC) que se assemelham a dados de captura- recapture reais. A abordagem GEE proposto ´eum procedimento de inferˆencia ´util para estimar parˆametros populacionais, especialmente quando uma grande propor¸c˜ao de indiv´ıduos ´ecapturada. Para uma propor¸c˜ao baixa de capturas, ´e dif´ıcil obter estimativas fi´aveis para todas as abordagens aplicadas, mas GEE su- pera os outros m´etodos. Os resultados das simula¸c˜oes mostram que o m´etodo da quase-verossimilhan¸ca do GEE fornece estimativas do erro padr˜ao menor do que o m´etodo da verossimilhana parcial dos modelos lineares generalizados (GLM) e GLMM. As estimstivas do tamanho da popula¸c˜ao variam de acordo com a na- tureza da correla¸c˜ao existente entre as ocasi˜oes de captura. Palavras-chave: Captura-recaptura experiˆencia; Estrutura de correla¸c˜ao; Equa¸c˜oes de estima¸c˜ao generalizadas; Modelos lineares generalizados mistos; Heterogenei- dade; Estimativa de tamanho da popula¸c˜ao; Crit´erio de informa¸c˜ao quasi- verossim- ilhan¸ca. Dedicated to Shadiq Muntaqim Akanda (Maahin) Maksuda Khanam (Shiuli) Wasif Tahsin Akanda Acknowledgement First and foremost I would like to express my heartfelt gratitude to my supervisor Professor Russell Gerardo Alpizar Jara who gave me the opportunity to work with him. His insights and knowledge have been extremely valuable. He motivated me how to write a paper step by step. His patience, guidance, active support, con- tinuous encouragement, amiability and generosity turned a difficult task into an immersing and enjoyable experience. During these years I have learned so much academic and non-academic things in a hospitable style. His door was always open for me right from the beginning. He is not only my supervisor, but also one of my closest best friends. It was my privilege to work with him. Thanks a lot for all the discussions and for the exhaustive text revisions indeed. I am profoundly grateful to him. I express my sincere gratitude to the European Union for awarding me the Eras- mus Mundus Mobility with Asia-West (EMMA-WEST) scholarship. I am also very grateful to my employer University of Dhaka, Bangladesh who facilitated my enrolment in this program. I feel that I have greatly enhanced my skills and broadened my outlook. The investment was worthwhile and I eagerly look to apply back my skills and contribute to my country. I am thankful to the Re- search Centre of Mathematics and Application (CIMA-UE) at the University of Evora´ and the Funda¸c˜ao Nacional para a Ciˆencia e Tecnologia (FCT), Portugal for providing some financial support to participate in conferences under the project PEst-OE/MAT/UI0117/2011. I am also greatfully acknowledge the comments x and suggestions made by jury members. I am utmost grateful to EMMA-WEST coordinators Professor Md. Shafiqul Islam of University of Dhaka, Professor Imme van den Berg and Professor Jos´eCarlos Tiago de Oliveira of University of Evora´ for their cordial support during my study period. Much appreciation is also due to the Rector at the University of Evora´ for nurturing a wonderful environment that made life here such a stimulating and enjoyable experience. My thanks goes to everybody in the Mobility and Interna- tional Relations Office, Department of Mathematics, Academic Services and Social Services at the University of Evora´ for their active support. Much of the credit of the dissertation goes to my beloved sons Shadiq Muntaqim Akanda (Maahin) & Wasif Tahsin Akanda and my heartiest wife Maksuda Khanam (Shiuli). They supported me in innumerable ways and continues to inspire and encourage me along in life. I would not be here without their love and support. My faith makes all things possible, but their love and support make all things easy. The list of sacrifices that they has made during this research would not fit on the pages of this dissertation.