University of Nevada, Reno Photometry+

Home , Photometry (astronomy)

University of Nevada, Reno

Photometry+: A Scientific Pipeline and Teaching Tool

A thesis submitted in partial fulfillment of the

requirements for the degree of Master of Science in

Computer Science and Engineering

Alexis R. Tudor

Dr. Sergiu Dascalu / Thesis Advisor

Dr. Richard M. Plotkin / Thesis Advisor

May, 2021

THE GRADUATE SCHOOL

We recommend that the thesis prepared under our supervision by

ALEXIS TUDOR

entitled Photometry+: A Scientific Pipeline and Teaching Tool

be accepted in partial fulfillment of the requirements for the degree of

MASTER OF SCIENCE

Sergiu Dascalu, Ph.D. Advisor Richard M. Plotkin, Ph.D. Co-advisor

Emily M. Hand, Ph.D. Committee Member

Melodi Rodrigue, Ph.D. Graduate School Representative

David W. Zeh, Ph.D., Dean Graduate School

May, 2021 i

Abstract

As more and more data are collected from the night sky, it becomes increasingly important to be able to analyze the data precisely and quickly by using computer programs. Given the importance of data analysis pipelines for telescopes we have developed a photometric pipeline, Photometry+, for the Great Basin Observatory (GBO), a 0.7-meter robotic telescope located in the Great Basin National Park in Nevada. This photometric pipeline takes raw images of the night sky and measures the magnitude of a star in the image. Studying the changes in the flux of a star over time is crucial for learning more about variable objects such as supernovae and binary star systems. Photometry+ focuses on human-computer interaction (HCI) in addition to scientific results. The HCI goals of the proposed pipeline are to create a graphical user interface (GUI) that is easy to use, gives astronomers control of and confidence in the results of the program, and teaches students the process of differential photometry through use. Our research shows that users agree that these goals are met and that just using Photometry+ results in a statistically significant 15% increase in score on a differential photometry quiz (from 66% to 81%). These results cement it as a new tool for professional astronomers looking to reduce the time they spend on data analysis while still obtaining publication-quality results and for students looking to learn the process alike. This thesis encompasses astronomy and human-computer interaction background, related works, software engineering aspects, practical astronomy uses of Photometry+, the user study research we completed, and the results we obtained. ii

Dedication

To my parents, Glenn and Judy Tudor, and my partner, Ryan Nunes for their endless support of me during the challenging times in which this work was created. Additionally I would like to dedicate this work to my Chihuahua, Lucky, who was there for me for most of my education and passed away in 2020. iii

Acknowledgements

I would like to thank from the bottom of my heart the advisors over this work, Dr. Sergiu Dascalu and Dr. Richard Plotkin, as well as Dr. Aarran Shaw, for their invaluable support and near-infinite knowledge, without which this research would never have come to completion. Additional thanks to the Great Basin Observatory, Nevada, for their support of this project and the use of their telescope, with particular thanks to Aviva O’Neil for all of her help. And finally, great thanks to the UNR Black Holes lab group, whose brainstorming ability and willingness to help was critical to making Photometry+ what it is.

Table of Contents

Abstract i

Dedication ii

Acknowledgements iii

Table of Contents iv

List of Tables viii

List of Figures ix

1 Introduction 1

2 Background 4

2.1 Photometry and Differential Photometry 4

2.2 Astronomy and Computing 6

2.2.1 Novae 7

2.2.2 Intermediate Polars 9

2.2.3 Black Hole X-Ray Binary Systems 11

2.3 Interactive Systems 12

2.3.1 User Experience Design 14

3 Related Works 15

3.1 Photometric Pipelines 15

3.2 User-Friendly Design in Scientific Software 16

3.3 Discussion 17

4 Photometry+: Software Engineering Aspects 19

4.1 Software Specifications 19

4.1.1 Discovery 19

4.1.2 Functional Requirements 21 v

4.1.3 Non-Functional Requirements 24

4.2 Technical Specifications 25

4.2.1 PyQt5 25

4.2.2 Astrophysics Libraries 26

4.2.3 VizieR, SIMBAD, and Astrometry.net 27

4.3 System Design 27

4.3.1 User Interface Design 29

4.4 Development 30

4.4.1 Test-Driven Development 30

4.4.2 Iterative Development 34

4.4.3 Development User Studies 36

4.4.4 Agile Development 36

4.5 Data Design 37

4.6 Documentation 38

5 Scenarios of Use 39

5.1 Code Version 39

5.1.1 Use Cases 40

5.2 Desktop Version 42

5.2.1 Use Cases 42

5.2.2 Home 44

5.2.3 New Project 44

5.2.4 Processing and Walkthrough Mode 48

5.2.5 Project Results 51

5.2.6 My Projects 52 vi

5.2.7 About 52

5.2.8 Settings 53

5.2.9 FAQ 54

6 Practical Astronomy Applications 56

6.1 Light Curve Analysis of Transient Nova AT2019tlu 56

6.2 Monitoring For Rare Low-Flux States of Intermediate Polars 58

6.3 Teaching Stellar Evolution with a Generated CMD 61

6.4 Studying the A0620-00 Black Hole X-Ray Binary System 62

6.5 Planned Observing Projects 64

7 User Studies 65

7.1 Participants 65

7.1.1 Age 66

7.1.2 Gender 66

7.1.3 Education Level 67

7.1.4 Operating System 68

7.1.5 Astronomy Experience Level 68

7.1.6 Photometry Experience Level 69

7.2 Apparatus 70

7.3 Procedure 70

7.3.1 Pre-Study Survey 71

7.3.2 Differential Photometry Quiz 72

7.3.3 Photometry+ Tasks 73

7.3.4 Post-Study Survey 74

7.4 Experimental Design 76 vii

8 Results and Discussion 77

8.1 Results of Post-Study Survey 77

8.2 Analysis of Dependent Variable Derived From Quiz 79

8.3 Discussion 79

9 Conclusions and Future Work 81

Appendix A 83

Appendix B 84

References 86

viii

List of Tables

Table 4.1: Functional requirements for Photometry+ versions, where C indicates the code version and D indicates the desktop application. 22 Table 4.2: Non-functional requirements for Photometry+ versions, where C indicates the code version and D indicates the desktop application. 24 Table 4.3: Example subset of tests used for test-driven development along with most recent results. 31 Table 5.1: This table describes the use cases in Figure 5.1 in more detail. 40 Table 5.2: Detailed descriptions of the use cases described in Figure 5.2. 43 Table 8.1: Results of post-study survey Likert scale questions. In all of the examples below, the scale starts at “Strongly Disagree” at one and ends at “Strongly Agree” at five. 78 ix

List of Figures

Fig. 1.1: The Great Basin Observatory, Nevada. Photo Credit: The Great Basin Observatory. 1 Fig. 2.1: Artist’s depiction of a cataclysmic variable, with white dwarf in the bottom left and the donor star in the top right. Art credit: Julie Bauschardt. 10 Fig. 2.2: The Xerox Star (left) and its GUI (right). Photo credit: The Interface Experience. 13 Fig. 4.1: The context diagram for Photometry+ displaying the external dependencies of the system. 26 Fig. 4.2: Abridged version of the steps Photometry+ takes to perform differential photometry. 28 Fig. 4.3: Light curve made with Photometry+ utilizing GBO data (top) matched with data from AAVSO, where the Photometry+ results are green and AAVSO results are blue (bottom). The match in the bottom figure demonstrates that Photometry+ works with a comparable accuracy to other top photometric pipelines. Figure generated by Ava Covington. 29 Fig. 4.4: A diagram showcasing the iterative development stages used for all parts of coding Photometry+. 34 Fig. 4.5: Three example screens of Photometry+ prototypes, with the desktop versions on top and the website version on the bottom. 35 Fig. 5.1: Use cases for the code version of Photometry+. 40 Fig. 5.2: Use cases for the desktop version of Photometry+. 42 Fig. 5.3: The Home screen of the desktop version of Photometry+. 44 Fig. 5.4: The New Project page of Photometry+ before any user input. 45 Fig. 5.5: The New Project page of Photometry+ after being filled out by the user. 45 Fig. 5.6: Demonstration of the tooltip associated with the catalog setting. 46 Fig. 5.7: Demonstration of the pop up window associated with clicking on the information button next to the catalog setting. 47 Fig. 5.8: Confirmation prompt after user tries to submit their project. 47 Fig. 5.9: Initial processing screen after a new project is submitted. 49 x

Fig. 5.10: Walkthrough mode reference star selection with all reference stars checked. 49 Fig. 5.11: Walkthrough mode reference star selection with Ref6 and Ref20 unchecked. 50 Fig. 5.12: Processing screen after reference stars are confirmed. 50 Fig. 5.13: Project results page for a DO Dra project. 51 Fig. 5.14: Project management page for Photometry+. 52 Fig. 5.15: The About page of Photometry+, broken into three of the scrollable sections in order. 53 Fig. 5.16: The default settings page of Photometry+. 54 Fig. 5.17: Frequently asked questions page of Photometry+. 55 Fig. 6.1: This photometry data shows the star AT2019tlu. This image is made of the data from the B, V, and R filters of the Great Basin Observatory, and AT2019tlu is circled. 57 Fig. 6.2: The light curve of AT2019tlu shows the peak of the nova and its rate of decline in the V filter. 58 Fig. 6.3: Calibrated image data from February 15th. DW Cnc is circled in green, and the reference stars used for photometry are circled in white. 59 Fig. 6.4: The light curve of DW Cnc showing magnitude over a period of a month and a half. 60 Fig. 6.5: CMD generated by Photometry+ from images, with stars selected for the lesson plan in red (left) and the CMD generated from values calculated by the students (right). 62 Fig. 6.6: Image of A0620-00 (circled) in the I-band taken by the GBO. 63 Fig. 6.7: Light curve of A0620-00. Figure generated by Matthew Neill. 63 Fig. 7.1: The example SIMBAD page that was provided to participants of the user study. 65 Fig. 7.2: Pie chart representing the age of participants in the user study. 66 Fig. 7.3: Pie chart representing the gender of participants in the user study. 67 Fig. 7.4: Pie chart representing the education level of participants in the user study. 67 xi

Fig. 7.5: Pie chart representing the primary operating system of participants in the user study. 68 Fig. 7.6: Pie chart representing the astronomy experience level of participants in the user study. 69 Fig. 7.7: Pie chart representing the photometry experience level of participants in the user study. 69 Fig. 8.1: Time it took for participants to set up their first Photometry+ project by astronomy experience level. 77 Fig. 8.2: Detailed information on the performance of participants on a differential photometry quiz before and after exposure to Photometry+. 79

1 Introduction

The importance of data analysis in astrophysics has become indisputable as data gathering techniques have gotten larger and faster. The spotlight has thus begun to shine on data science and software development as key supporting fields of astrophysics. Given the importance of data analysis pipelines for telescopes of all kinds, we have developed a photometric pipeline, Photometry+, for the Great Basin Observatory (GBO), a 0.7-meter robotic telescope located in the Great Basin National Park in Nevada (GBNP Foundation, 2020). The GBO is the first research grade observatory located in a U.S. National Park, shown in Figure 1.1, enjoying dark skies free of light pollution. It partners with universities in order to inspire researchers and students alike to enjoy astrophysics, and has been shown to be a great candidate for automation with other Python processes (Fausett et al., 2019).

Fig. 1.1: The Great Basin Observatory, Nevada. Photo Credit: The Great Basin Observatory.

Photometry is the process of measuring the flux of celestial objects in astronomical images. Therefore the goal of a photometric pipeline is to analyze these raw images of the night sky that are taken by counting relative photons hitting a charge coupled device (CCD) to calculate the flux of this object. The flux of an object is measured on a magnitude scale, wherein the brightest stars consist of smaller numbers (the Sun being - 26 magnitude) and each step in magnitude represents a logarithmic decrease in brightness (with Pluto having a magnitude of around 15). The measurement of fluxes represents one of the most basic deliverables from astronomical images, which in most cases is one of the key pieces of information to determine the physics driving astronomical phenomena. For example, supernovae represent exploding stars, which are discovered via the sudden appearance of a bright source in the sky. Measuring Type Ia supernovae in this way allows astrophysicists to calculate the large stellar distances, which is instrumental in the field of cosmology (Riess et al., 1998). Or, monitoring changes in the flux of a star over time can (sometimes) indicate changes in its surface temperature and/or radius (Madore and Freedman, 1991), provide information on the orbital parameters of certain binary systems (Andersen, 1991), or assist in the discovery of exoplanets using the “transit method” by detecting dips in the light coming from stars (Deeg and Roi, 2018). However, analyzing the raw data requires several steps that can be tedious to do by hand. For each sky image there are an additional two to three calibration files that are required to remove noise and intrinsic pixel-to-pixel variations from the raw CCD images. Additionally, cosmic and terrestrial background radiation needs to be subtracted by taking a median of photons counted from “blank” sky. Once the error is reduced by removing noise, the magnitude of a target object can be calculated by comparing it to other stars of known magnitude in the same image and taking an average of the results calculated for each comparison, which is a subtype of photometry called differential photometry. The process of comparing a target object to multiple reference stars can take up to a half hour per image to do by hand, and is described in more depth in Section 2.1. However, by automating the process Photometry+ can perform these same tasks in less than 20 seconds per image.

To prevent Photometry+ from being a black box tool, we focus on the human-computer interaction (HCI) components of the program. The HCI goals of the proposed pipeline are to create a graphical user interface (GUI) that is easy to use, gives astronomers control over the program, increases confidence in the results of the program, and can be used to teach students the process of differential photometry as described above. To validate the accomplishment of these goals three user studies were conducted, two of which have been used to guide the development of Photometry+, and a final user study to validate that Photometry+ can be used as a teaching tool for the complex task of differential photometry. The first two user studies tested how long it takes inexperienced and experienced astronomers, respectively, to use the GUI to complete the task, the parts of the process they found confusing, their confidence in the tool, and how much they feel they learned about differential photometry through using the tool.

Using the feedback received from the first two user studies, a public-release version of Photometry+ was developed and the final user study was completed with this version. The final user study tests the hypothesis that user-guided development of a GUI can be used to create scientific tools that are useful for (i) experienced astronomers completing routine results generation, and (ii) beginners looking to learn more about the process. User confidence in the results was measured both in terms of the control users feel they have over the program, and in their confidence in the accuracy of the results obtained. We also present examples of the application of Photometry+ for monitoring variable objects. Photometry+ provides a new look at photometric pipelines with the user and HCI principles in mind. This user-oriented design allows Photometry+ not just to be a tool for experienced astronomers, but also a teaching tool for students and others looking to learn differential photometry. While Photometry+ was designed with a specific scientific process target, this approach is generalizable enough to be used on any tool seeking to teach a difficult scientific concept through performing a task.

2 Background

Photometry+ and the research contained in this document span a wide variety of topics in both astrophysics and human-computer interaction. This section will provide a background on the scientific and computing topics included in this document.

2.1 Photometry and Differential Photometry

When it comes to monitoring a variable object over time, a light curve is a valuable tool in the astronomer’s arsenal. A light curve is a graph of the flux coming from a star over time and is a way to detect changes in the magnitude of a star. The process of analyzing light in the night sky is called photometry, which was briefly covered in Chapter 1. Photometry and photography share a common root due to their common medium: light. However, due to the measurement precision required for astronomical analysis, the process is more difficult than conventional photography. Modern photometry is digital and uses a charge-coupled device (CCD) that consists of a grid of ‘pixels’, usually numbering in the thousands, which are designed to produce signals that are proportional to the number of photons incident on each pixel. We henceforth refer to these signals as counts. To take photometric data the CCD is pointed at the part of the sky where data needs to be acquired, and allowed to face that sky for a certain amount of time. This exposure time is a balance between capturing as many photons as possible to get a more detailed image of the sky, including fainter targets, and leaving it exposed too long and risking saturating the detector.

Photometry data is not immediately usable. When the CCD collects counts, it also collects background radiation, counts that are ‘stuck’ in the CCD, and dead pixels. To solve this problem the image data is calibrated with three kinds of telescope images. A bias frame is taken with an exposure time of zero and with the shutter on the camera closed. This frame accounts for read-out errors, dead pixels, and other errors inherent to the CCD. The bias frame also records the artificial signal injected into the CCD, an amplifying signal that prevents negative or low counts in the image. The second frame is 5 the dark frame, which is taken with the shutter closed for the same exposure time as the image. This removes thermal noise generated by the detector. The final frame is the flat field, created by illuminating the CCD with a uniform light source that allows one to correct for varying quantum efficiencies of pixels across the detector. However, the flat field also needs to be calibrated before use. The flat field must have the bias level and thermal noise measured from the bias/dark frames subtracted from it and then that result is normalized (the Flat in equation (2.1) refers to this calibrated and normalized flat field). While bias and dark frames are subtracted directly from the raw image, the flat field is divided from it as shown in equation (2.1). It is worth noting that the information on the bias frame is included in the dark, and thus the bias frame needs to be subtracted from the dark frame before both are subtracted from the raw image.

Calibrated = (Raw – Bias – Dark) / Flat (2.1)

Additionally, not all photometric methods are the same. The methodology outlined in this thesis, however, refers to differential photometry and from this point onward the use of the word ‘photometry’ will only be in reference to this method. Differential photometry is the process of finding the magnitude of a star by comparing it to multiple stars of known magnitude in the same image as a target object. This way, any atmospheric interference or telescope problems likely equally affect both the target object and the reference stars, allowing an accurate measurement to be made.

In order to calculate relative magnitude, we must be able to get an accurate relative count of the number of photons that hit the CCD from the locations of the stars. However, the counts in the area around a star are not only attributable to the star itself, but also include some amount of cosmic background radiation and terrestrial light from Earth scattered throughout the atmosphere. This background is subtracted out from the counts in the star based on the amount of counts coming from “blank” parts of the image (i.e. the areas of the image with no visible stars or objects in them). Using this information it is then possible to calculate the accurate number of counts in both the target object and all 6 reference stars and use that information to obtain the relative magnitude of the star using equation (2.2). In this equation N is the number of counts captured in the star (Nsrc being the number of counts in the target object), and m is the magnitude of the stars (msrc being the target object and being calculated for every reference star i).

(msrc)i = -2.5log10 + mi,ref (2.2)

This calculation is done for up to as many reference stars as can be found in the image, in which Ni,ref is the number of counts in a specific reference star and mi,ref is the magnitude of that star. Then all of these magnitudes are averaged together as the final measurement of the magnitude of the target object. The error for this calculation can then be calculated in a variety of ways, with the most common method being the standard deviation.

2.2 Astronomy and Computing

Astronomy is widely considered the oldest form of science in existence, with the first recorded astronomical observations dating back to the first form of human writing, cuneiform in ancient Mesopotamia (Shuttleworth, 2021), and it is likely that astronomy itself predates our ability to record our observations. To cover the history of astronomy in its entirety is a daunting task that would be best served in a paper written by an astronomer or historian, and so this background section will focus on an abridged version that is relevant to this thesis.

The first photometric measurements were done without tools and relied on measurements made by the human eye. The earliest records of these measurements come from Greek astronomers in 130 BCE, where an astronomer named Hipparchus is attributed with inventing the magnitude system for measuring the relative brightness of the stars (Britannica, 2021). This labeled the brightest stars as being of first magnitude, followed by second magnitude, and so on and so forth, with smaller numbers indicating brighter stars. This system is still in place today on a logarithmic scale implemented by Norman Robert Pogson (Pogson, 1856). The invention of photography brought even more 7 accuracy to these measurements, with the first permanent record photometry being done on photographic plates that could then be compared to other plates to establish relative magnitudes for stars in a semi-permanent record (Landau, 2019). And as photography improved, so did photometry. Now we use the CCD photometry methods outlined in Section 2.1 and at its advent the bounty of data that could be taken started to wear on the ability of human operators to process it all.

Almost as soon as computers were invented, astronomers leaped on the ability to use them to process their data. In the 21st century Python, the up and coming high-level programming language that was simple to learn and strong enough for image processing, has become the premier language for astronomical processing. In the beginning of using Python though, functions for performing astronomical processing were scattered across the community. Each lab had its own set of functions designed to do the same things. However in the early 2000s an open source collaboration created Astropy (The Astropy Collaboration, 2013), a universal library for astronomy programming. Astropy made a united set of functions that are generalized to be used by any astronomers. That is where astronomy and computing stand now.

Wide uses of photometry, astronomy, and computing are covered in Chapter 3, however Photometry+ has been used for specific stellar objects. These stellar objects are described in the following sections so that the science behind them can be understood while discussing the application of Photometry+ in Chapter 6.

2.2.1 Novae

When stars with similar masses to our Sun die, they leave behind ‘corpses’ known as white dwarfs. As Sun-like stars expend their primary fuel, Hydrogen, they start to fuse heavier elements until the star reaches a point where it can no longer complete nuclear fusion for less energy than it produces. These stars will expand into red giants before shedding its outer layers entirely. Once those layers have been shed, all that will be left of the star is its bare core, which has at this point been fused into helium (for less massive 8 cases) or carbon (for more massive cases). This core, which at this point is what astronomers call a white dwarf, is not hot enough to continue fusion, and therefore gravity causes it to become compressed to a very high density. Eventually the white dwarf becomes so dense that it requires a force to support itself from collapsing due to its own gravity. This force comes from the Pauli Exclusion Principle. This states that no two electrons can inhabit the exact same quantum state at the same time. Once a low energy level is filled, more electrons are forced to occupy higher energy states and move progressively faster. This energy causes electron degeneracy pressure, which is the force keeping white dwarfs from collapsing in on themselves from the force of gravity (Chandrasekhar and Milne, 1931).

Because of this pressure, white dwarfs have an inverse relationship between their size and their mass; if one were to add more mass (which only happens in binary systems where matter can accrete onto the white dwarf from another star), the star would get smaller until it reached the Chandrasekhar limit, 1.4 solar masses, where the electrons in the white dwarf approach the speed of light, causing the white dwarf to become a supernova (Chandrasekhar, 1931). The other kind of stellar explosion a white dwarf can undergo is more common. When a white dwarf accumulates mass, usually by stealing from a binary companion star, the mass can accumulate as a non-degenerate layer on the surface of the star. This mass gets hotter and hotter on the surface before experiencing a thermonuclear reaction with less force than a supernova. This event is called, appropriately, a nova (Shen et. al., 2018). A nova, while less bright than its cousin the supernova, increases the white dwarf’s flux by several magnitudes, making a previously unobservable target visible to smaller telescopes. Additionally, unlike the destructive supernova, the nova leaves the white dwarf entirely intact and surrounded by another layer of gas and dust reminiscent of the layers left behind when the star became a white dwarf. This process can continue as long as the white dwarf does not accumulate 1.4 solar masses of mass in its core.

For transient novae like the nova covered in Section 6.1, photometry is an essential part of learning about the star. Photometry allows scientists to trace the bright explosion of the accreted layer of material over time and learn more about both the star and other stars like them. Additionally, explosions like these are important for understanding how certain heavy elements, such as iron, proliferate in the universe, and thus allow for life to form.

2.2.2 Intermediate Polars

Cataclysmic variables (CVs) are variable objects describing binary star systems containing a white dwarf and another star, including those discussed in Section 2.2.1 (an artist’s depiction of an intermediate polar, which is a subclass of CVs, is shown in Figure 2.1). When material strays outside of the Roche lobe (which is the area around a star in which matter is gravitationally bound to it) of the larger partner star, it ‘falls’ onto the white dwarf, accumulating to form an accretion disk around the white dwarf (Bath, 1977). Magnetic CVs (mCVs) are CVs where the white dwarf part of the binary system has strong magnetic fields (about 106 - 108 G) that form ‘accretion curtains’ along magnetic field lines. This can manifest two ways. In polars the accretion disk is completely disrupted and the spin period of the white dwarf is synchronized with the binary orbital period. In intermediate polars (IPs) the magnetic field is not strong enough to disrupt the entire accretion disk, only influencing the center of the accretion disk around the white dwarf. Due to this magnetic field, the spin period and binary orbital period are not synchronized (Ustyugov, 2013).

By nature, both kinds of polars are variable in flux. A part of that is that reductions in accretion rates can result in dramatic decreases in optical and X-Ray flux. This is a “low- state”, which we define as a significant drop in flux (about one magnitude) below the average for that CV. These low-states are so extraordinarily rare that only two have ever been observed in X-Rays: FO Aqr (Kennedy et al., 2017) and V1223 Sgr; and only FO Aqr has a deep, pointed X-Ray observation. There are some theories as to what causes these decreased flux states, however with so few observations it is difficult to say for sure 10 why it happens. Looking at the data, it is clear that pointed X-Ray observations of low- state IPs are needed to be able to understand these objects.

Fig. 2.1: Artist’s depiction of an intermediate polar, with a white dwarf and accretion disk in the bottom left and the donor star in the top right. The inner part of the accretion disk is disrupted, indicating a magnetic white dwarf. Art credit: Julie Bauschardt.

As mCVs are subclasses of CVs, and IPs are subclasses of mCVs, IPs have a subclass known as low-luminosity IPs (LLIPs) that demonstrate X-Ray luminosities about two orders of magnitude fainter than most other IPs. These LLIPs are already very low flux, and their luminosity already lies below even the low state of FO Aqr. Therefore, it was not expected that LLIPs could have low states where they transition to even lower flux. However in June 2017 the LLIP DO/YY Dra was found to be in a low state, proving that it is possible (Andronov et. al., 2017).

Recording an IP in a low state with an X-Ray telescope will double the amount of deep- pointed X-Ray observations of low-state IPs, and recording an LLIP in a low state will be the only observation of its kind. This could give astrophysicists new insight into accretion mechanisms, a process that is poorly understood due to a lack of data. However, it can be difficult to know where to point the X-Ray telescope at the right time. Therefore quick photometric monitoring of possible LLIPs is important in order to know when an LLIP enters a low state to take X-Ray measurements of it.

2.2.3 Black Hole X-Ray Binary Systems

An X-Ray binary system is similar to the cataclysmic variables described in Section 2.2.2 in that it is a system where matter is siphoned from a companion star onto its partner. The difference in this lies that rather than a white dwarf siphoning material, in an X-Ray binary it is usually a neutron star or a black hole. These systems produce some of the brightest X-Ray signatures in the galaxy, making black hole X-Ray binaries much easier to locate than their lonely black hole counterparts (Kohler, 2018). These systems are usually sorted into two main categories based on the mass of the donor star: high-mass X- Ray binaries (HMXB) which are over two solar masses in size, and low-mass X-Ray binaries (LMXB) which are under two solar masses. These systems can shed a lot of light on black hole physics, a field of physics that still lies on the border of our understanding of the Universe.

When monitoring these black hole X-Ray binaries, there can be two parts to observe. The first observable part is the donor star itself. The donor star will be visible at optical wavelengths and will be rotating around the black hole. It is worth mentioning that although this is the case for the black hole X-Ray binary mentioned in this document, the donor star is not always visible (particularly in cases where the accretion disk of the black hole is very bright). The second observable part is the accretion disk containing the matter that has been siphoned from the star and is currently orbiting inside of the Roche lobe of the black hole. The black hole is, like others, not directly observable except for this accretion disk. Parts of this accretion disk emit at optical wavelengths, though as the 12 material approaches the black hole it emits in X-Ray. In most research the accretion disk around the black hole is the more interesting part of the observations, so removing the orbital donor star is an important part of the process of studying them.

2.3 Interactive Systems

Interactive Systems are, loosely, any computer system that users interact with. This can refer to anything from operating systems, like MacOS or Windows, to individual applications. In fact, if you are not reading this on a physical piece of paper, you are using an interactive system right now. At the time this thesis is being written, interactive systems design is a large part of making anything for the computer and we as a society have grown accustomed to a certain standard for ease-of-use. However, computers were not always this easy to use.

Some of the first interactive systems ran mostly on a command line-based interface that required a lot of command memorization and had strict standards for input and output. These systems put a great burden on the user to make sure that everything was correct and did little to minimize user mental load or errors. After it became clear that command line systems were prohibitively difficult for some users, another generation of interface was created based more on menus and dialogs. This required less memorization of commands and instead allowed users to select options they would like to use rather than type them in themselves. These systems were simplistic and overly relied on the user’s ability to memorize, repeat, and order their information.

The first interactive system that had the trimmings of a modern computer was the Xerox Star (Xerox Star, n.d.), which was released in 1981 and shown in Figure 2.2. While not enjoying a large commercial success, the Xerox Star was the first device that brought the computer mouse and a GUI to the public. Although it is difficult to picture now that they are staples of modern computing, these developments were revolutionary at the time. Using the mouse, users could directly interact with the things they saw on the screen 13 rather than type out everything. Although the Star did not achieve large success, it paved the way for the computer systems that came after it.

Fig. 2.2: The Xerox Star (left) and its GUI (right). Photo credit: The Interface Experience.

This huge leap from command-based interfaces to direct and visual interfaces was the beginning of putting computing in the hands of ordinary people. The Xerox Star was followed by Apple’s Macintosh computer and operating system, which was then shortly followed by Microsoft’s Windows system. These computers were designed for home use and quickly became popular, and now it is unusual not to see at least one such device in every home. A large focus has now been placed upon improving these direct interfaces in order to make interacting with a computer and its programs easier, faster, and more enjoyable (Interactive Systems, 2021).

In addition to the standard mouse-and-keyboard computers, many different methods of interacting with computer systems have been designed. The primary focus of this thesis is a computer system, and thus this section shall be brief, but it is worth mentioning the many other kinds of interactive systems available today. Video game systems, interacted with via a multitude of different controller types, have become increasingly prominent. Additionally, video game development has led to the creation of virtual and augmented reality interfaces that allow users to interact with systems using virtual reality “goggles” 14 that rest on their face. Another very common type of interface is the touch screen device, like tablets or phones. Touch screens are also occasionally applied to computers themselves in addition to the mouse and keyboard. And these are only a few novel interactive systems, and many more are in development.

With the widespread use of computers and other interactive systems, a new field of computer science has come onto the world stage. User experience design, which will be covered in Section 2.3.1, focuses on improving the interactive system for use.

2.3.1 User Experience Design

User experience (UX) design is a relatively new field that is intrinsically tied to the creation of better interactive systems from a user perspective. UX design is, defined simply, the science of designing a product to provide the best possible experience to the user. This involves all sections of design, not just the design of user interfaces. UX design involves user guides, product feel, ease-of-use, acquisition, and all other features that concern the end user. The way the product looks, feels, is used, and is acquired are all under the domain of a UX designer. Large programs, websites, and other systems that are known for their excellent user interfaces often hire a UX designer or a team of UX designers. UX design principles were used in the development of Photometry+ including its use, documentation, logo design, and overall design.

3 Related Works

The age of big data has changed the way many fields are able to operate, including astronomy and astrophysics. Telescopes around the world produce a colossal amount of data, with some telescopes producing data in the Exabyte range (Zhang and Zhao, 2015). It is only natural that the large amount of data needing to be processed has put an emphasis on data analysis pipelines of all sorts.

3.1 Photometric Pipelines

One kind of data analysis pipeline, the photometric pipeline, focuses on performing different kinds of photometry on telescope images to calculate the fluxes of stars. Most of these photometric pipelines are not generalized, but rather built with a single telescope or telescope system in mind (with only a few exceptions Mommert, 2017). Some of these pipelines are open source and, although they are designed for a specific telescope they allow for other researchers to use modified versions of their pipeline. An example of this is the Vera C. Rubin Observatory Legacy Survey of Space and Time (Rubin’s LSST) Science Pipeline (Ivezić 2009), which is designed for the Vera C. Rubin Observatory but whose code is available and modifiable for anyone interested in it. One of the more complex photometric pipelines available is designed for the All Sky Automated Survey for SuperNovae (ASAS-SN), which is not a single telescope but a network of telescopes designed to work together to image the entire night sky (Kochanek, 2017). The ASAS- SN photometric pipeline is accessed through a web portal that allows users to find a light curve of anywhere in the night sky, assuming that there is data for that location on the sky at the time when the user wants to observe. However, large telescopes and surveys are not the only systems with automatic pipelines. Smaller telescopes for different purposes, like the Watcher robotic telescope in Boyden Observatory, have photometric pipelines designed for them (Ferrero, 2010). Some photometric pipelines are even designed with a backlog already in mind, such as the pipeline created for the Robotic Optical Transient Search Experiment (ROTSE)-IIId archival data (Güçsav, 2012). That photometric pipeline is used almost exclusively for archival data, though that is not always the case 16 for pipelines designed to handle archival data. The pipeline for the Wide Field Astronomy Unit (WFAU) is built to parse data fast enough to continually process new data in addition to processing archival data that has backed up (Cross, 2010). Clearly the creation of photometric pipelines for telescopes around the world is widespread, and with Photometry+ there is now a new pipeline for the GBO telescope as well (Tudor, et. al., 2021).

While the general goal of all photometric pipelines is to perform photometry on images from telescopes, many pipelines are made with additional goals in mind. For instance, some pipelines are designed to cater to specific types of stellar objects rather than a telescope. One such example, the Pippin pipeline, is an open source pipeline designed for supernova-based analysis (Hinton, 2020). Other pipelines are built with specific observing methods and specific types of stellar objects, such as the Pulsar Arecibo L- Band Feed Array (PALFA) single-pulse pipeline focusing on finding single-pulse objects like pulsars and radio transients (Patel, 2018) (though PALFA is a radio telescope, and not an optical telescope). However, not all of the pipelines with additional goals are focused on certain stellar objects. Some of these pipelines instead direct their attention toward data quality or other mechanical parts of the astrophotography process. One such example is a pipeline that uses a convolutional kernel to reduce the effect blurry images have on the final photometric calculation (Huang, et. al., 2020). Like these other pipelines, Photometry+ includes more than the standard goal of performing photometry. Unlike these other pipelines however, the additional goal of Photometry+ does not focus on space or data correction, but rather on the human element of interacting with the pipeline.

3.2 User-Friendly Design in Scientific Software

Scientific software and usability have always had a complicated history. Software developers can often be entirely absent when it comes to making the computational tools that scientists use on a daily basis. Thus, good design practice can often be neglected. This problem goes back to the early days of software being used as a scientific tool with 17 observers noting that the creation of user interfaces (UIs) for scientific tools are ill- funded, poorly understood, and less emphasized (Chiozzi, et al., 2007). And although it is not a focus in scientific software, usability-centered design can have many benefits including reducing user errors, reducing the time it takes to learn to use a tool, and making software more generalizable. Adding user-centered design principles to scientific software does not have to be difficult either, as studies have shown beginning the process does not take very long and brings many benefits (Macaulay, et. al., 2009). In recent years usability has become a focus of some astronomy developers, such as in visualization software for radio astronomy (Rampersad, et. al., 2017). Rampersad et al. used user-guided development to create their visualization tool, holding user studies in between prototypes and molding it to be more user-friendly and easy to use. Another example is the European Southern Observatory (ESO) which develops intuitive user interfaces for some of their astronomy tools, such as the web portal for their phase 2 material (Vogt, 2018). While there are examples of user interface design principles being used in designing astronomy software, we argue that UX design should be used more broadly than it is now.

We also argue that this can be taken a step further by combining these development user studies with research studies that validate what a pipeline can do for the user. Pipelines can be more than just user-friendly; they can be a valuable teaching tool for students looking to learn complicated scientific processes. Photometry+ is a new step in the combination of HCI and astronomy, as a tool that obtains high quality results and is easy to use. It is also a teaching tool for those looking to learn about differential photometry.

3.3 Discussion

From the history of astronomy and computing reviewed in Section 2.2, we see that a pattern of consolidation occurs in astronomy software. When technology changes or new technology is created, oftentimes the start of processing with that new technology is fractured across different labs and telescopes. That is what we see now, with photometry pipelines often in use by only a single lab or created for a single telescope. However it 18 can be predicted that sometime in the future a universal photometric pipeline library will also be created, and when it does, we argue that UX design and HCI should be a critical part of it. That is why Photometry+ seeks to contribute to the general body of work that uses UX design techniques in scientific software. 19

4 Photometry+: Software Engineering Aspects

Photometry+ is a photometric pipeline that performs differential photometry using Python. There are two versions of Photometry+. The first version is a code version that is raw Python code and can be run in the Python terminal. This version makes up the back end of the second version. The second version is the desktop version of Photometry+. The GUI on the desktop version makes the program accessible even to inexperienced users and is focused on making differential photometry easy to use and then learn.

4.1 Software Specifications

Photometry+ was designed with the help of astrophysicists working at the University of Nevada, Reno and at the Great Basin Observatory. These astrophysicists provided feedback in all parts of the process, as explained in Section 4.4.2, however to begin the process a normal waterfall method-type requirements discovery was used. Using this discovery process functional and non-functional requirements were established. Initially during development, these functional and non-functional requirements were prioritized based on their likelihood of being developed, but as of the end of development all requirements were fulfilled.

4.1.1 Discovery

To discover the requirements for Photometry+, three stakeholders currently working in the astrophysics field were sent an electronic survey asking what features they would like to see in Photometry+. Additionally, some of the surveyed stakeholders had used a beta version of the program and were asked for their experience with the program.

Questionnaire: 1. What is the highest level of education you have completed? a. Did not attend school b. Some school c. Graduated from high school d. Some college e. Graduated from college 20

f. Some graduate school g. Completed graduate school 2. What best describes your current interest in differential photometry? a. Student b. Researcher c. Industry Professional d. Other (please specify) 3. What level of experience do you have with manual differential photometry? a. Professional experience b. Moderate experience c. Some experience d. No experience 4. What is your level of proficiency with the computer programming language Python? a. Completely proficient b. Moderately proficient c. Some experience, not proficient d. No experience 5. What are the most important things for someone about to perform differential photometry to know? 6. What functions would you like to see in a program designed to perform differential photometry? 7. How would you like to access a program for differential photometry? a. Through a web site b. Through a downloadable desktop application c. Through the download of code on GitHub d. Other (please specify) 8. Have you used other astronomy tools, such as Astrometry.net or DS9? What did you enjoy about them? 9. What did you dislike about them? 10. If you have used Photometry+, please give feedback on your experience. What was done well? What can be improved? Was there any difficulty in using the program?

Summary of Questionnaire Answers In terms of demographics, the stakeholders were all college educated, ranging from undergraduate students to PhD holding researchers. The stakeholders each had the level of experience with astronomy that would have been expected for their education level. All subjects said they were moderately proficient with Python. Important things to know about differential photometry that were mentioned frequently were how to calibrate files, choosing appropriate reference stars, and the fine details of how magnitudes and the 21 scientific background works. The following list of requested features was made based on answers to question six: 1. Functions to search through directories for images 2. Visualization options 3. Ability to change aperture sizes manually 4. Balance between automation and interaction a. User should choose aperture sizes, background regions, and reference stars, then switch to automated 5. Identify bad images 6. Easy-to-read results 7. Plot a light curve

When asked how they wanted to use the program, the most requested access to the code was via GitHub (Tudor, GitHub, 2021), with preference then equally divided between a desktop application and a website. When compared with other common astronomy tools, stakeholders expressed that they most admired visualization features in other astronomy tools, though they found that the other tools were difficult to use and sometimes could not process multiple images at once. Those stakeholders who had used the beta version of Photometry+ had few critiques, and those critiques were easily addressed.

4.1.2 Functional Requirements

Functional requirements specify how the system shall behave, what it can and cannot do, and its basic functionalities. In Table 4.1 the functional requirements for Photometry+ are outlined, and the version each functional requirement applies to is indicated. Generally, the code version of Photometry+ has stronger capabilities, however it is much harder to use than the desktop version, which will be evident in Section 4.1.3.

Table 4.1: Functional requirements for Photometry+ versions, where C indicates the code version and D indicates the desktop application. Requirement Version Description

FR01 C & D The program shall be able to subtract dark frames and bias frames, and divide flat fields from a raw FITS file.

FR02 C & D The user shall have the option to subtract bias frames from dark frames before calibrating

FR03 C The user shall have the option to output the calibrated FITS information to a new file.

FR04 D The user shall see the calibrated files as they are processed.

FR05 C & D The program shall be able to calculate the median counts per pixel in a “blank” sky aperture of predetermined or chosen size.

FR06 C & D The program shall be able to pull the full-width half- maximum (FWHM) from the header of a FITS file if it exists, and not crash the program if it does not exist.

FR07 C & D The program shall be able to tally the counts in pixels of a given area.

FR08 C & D The program shall be able to subtract background counts for every pixel in a given area.

FR09 C & D The program shall access user-specified VizieR catalogs or the SIMBAD database to find magnitudes of stars in the image.

FR10 C & D The program shall truncate printed values to an appropriate length.

FR11 D The program shall summarize results in an understandable way.

FR12 C & D The program shall document its results and findings by creating new files.

FR13 C The user shall be able to change data in the files for use by the program.

FR14 C & D The program shall be able to read in files and incorporate the data into its calculations. 23

FR15 C & D The program shall be able to calculate the magnitude of a star given appropriate reference stars with counts and magnitudes, either autonomously found or entered.

FR16 C & D The program shall be able to calculate the error of the final magnitude calculation in multiple ways.

FR17 C & D The program shall be able to attach new WCS data to a FITS file using Astrometry.net given an internet connection and a valid Astrometry.net API key.

FR18 C & D The user shall be able to choose different settings that change how the program runs.

FR19 C & D The program should automatically discard unusable images.

FR20 C & D The program shall be able to pull all files from a folder directory and run them through photometry one after the other.

FR21 C & D The program shall put all created files in a new folder.

FR22 C & D The program shall be able to find the radius of a target object by analyzing the image itself.

FR23 C & D The user shall be able to choose to use a weighted magnitude to reduce error.

2 FR24 C & D The program shall be able to calculate the � r value of a set of measurements.

FR25 C & D The program shall detect if any part of the WCS data is missing from the FITS file.

FR26 D The program should allow the users to see and choose reference stars to use.

FR27 D The user should be able to use a menu to adjust settings for the program.

FR28 D The program should provide updates on what it is doing as it runs images for photometry.

FR29 C & D The user should be able to select which filter to use for magnitude calculation.

FR30 D The user shall be able to choose to interact with the program 24

or to have it just run.

FR31 C & D The user shall be able to choose to use median background counts for the area in which the star resides instead of generally over the whole image.

FR32 D The user shall see the FITS file uploaded during processing.

FR33 C & D The program shall automatically detect if a second star is in the radius of a reference star and remove that star from consideration.

FR34 C & D The program shall automatically remove outlier reference stars based on z-score analysis.

FR35 C The user shall be able to generate a CMD based on all stars in an image.

FR36 D The user shall be able to read about the program and its creators.

FR37 D The user shall be able to access frequently asked questions.

4.1.3 Non-Functional Requirements

Non-functional requirements of the program deal less with what the program does, like functional requirements, and more to do with how the program does it. These are intangible things, like use time, security, and how the program works under different circumstances. In this area the desktop version of Photometry+ provides more ease-of-use non-functional requirements than the code version. The non-functional requirements of Photometry+ are in Table 4.2, with the version each requirement belongs to indicated.

Table 4.2: Non-functional requirements for Photometry+ versions, where C indicates the code version and D indicates the desktop application. Requirement Version Description

NFR01 C The program shall pass every test case before any release.

NFR02 D The program shall be able to run in a desktop application 25

NFR03 D The program shall take no more than a half hour for the average user to be able to run photometry for the first time.

NFR04 C & D The program shall not run longer than a minute without telling the user what it is doing.

NFR05 C & D The program shall minimize catastrophic user errors by not changing or deleting original data.

NFR06 D The program shall minimize user errors with messages confirming actions before they are completed.

NFR07 C & D The program will be able to work in a limited capacity without the internet.

NFR08 C & D The program shall do photometry with a returned error of less than 0.2 on a majority of cases.

NFR09 D The program will use professional graphics.

NFR10 D The program will use HCI principles to make a user interface that looks clean and feels good to use.

NFR11 C & D The program will be generalizable to any telescope data.

NFR12 D The program shall teach differential photometry to those unfamiliar with its process.

4.2 Technical Specifications

Photometry+ uses Python as its development language due to its popularity with the astronomy community and the easy access to astrophysics libraries. Its popularity ensures that the most targeted end users will be able to understand and use the code version of the code and that many will be able to understand it enough to adapt it for their own use. Additionally the abundance of astrophysics libraries, highlighted in Section 4.2.2, provide easy access to complicated processes required for the differential photometry experience.

4.2.1 PyQt5

The desktop version of Photometry+ was created using the open source licensed version of PyQt5 and the QtCreator. The first version of the Photometry+ desktop version used 26

Tcl/Tk for its GUI, however Tcl/Tk performs poorly on MacOS and therefore that version was scrapped. The PyQt5 components of the GUI are stored as .ui files created and edited with the QtCreator.

4.2.2 Astrophysics Libraries

To create Photometry+ and allow it to be robust required several external resources. Like many other astronomy tools, Photometry+ used the Astropy library (The Astropy Collaboration, 2018), photutils (Photutils, 2020), and DAOPHOT (Stetson, 1987). Additionally, Photometry+ pulls information from the APIs for Astrometry.net (Lang, 2010), VizieR, and SIMBAD. These external dependencies are shown in the context diagram presented in Figure 4.1.

Fig. 4.1: The context diagram for Photometry+ displaying the external dependencies of the system. 27

4.2.3 VizieR, SIMBAD, and Astrometry.net

In addition to the astrophysics libraries built into the program, Photometry+ also uses the API services of several other astronomy tools. In order to find reference stars of known magnitude, Photometry+ uses the API of the VizieR system of astronomy catalogs (Ochsenbein, et. al., 2000) and the SIMBAD astronomical catalog (Wenger, 2000). The SIMBAD API can be accessed for any image, and can be used by users that do not have a specific catalog in mind (although variance in SIMBAD’s records can introduce error into the measurements). The VizieR API requires a specific VizieR catalog to be correctly formatted and entered, and in order to maintain consistency and reduce the final error of our measurements we use the APASS catalog (APASS, 2021) as the default VizieR catalog.

For telescope images that do not have world coordinate system (WCS) data, Photometry+ connects via API to the Astrometry.net system for astrometry of telescope images. This allows Photometry+ to expand the usable data it gets and also to be able to take in images of different kinds, including JPEG or PNG images (though FITS is still preferred). Using Astrometry.net to amend WCS information to images requires the user to input a valid Astrometry.net API key in order to both use the API and connect their images to their Astrometry.net account.

4.3 System Design

Photometry+ performs all the stages of differential photometry with minimal user input. Figure 4.2 shows a flowchart representing the steps of differential photometry performed by the program. To begin, users simply need to upload a telescope image and the stellar coordinates for the target object whose magnitude will be calculated. At this stage users can also optionally add calibration files for automatic calibration of their images and change the default settings of the program. Examples of the settings that can be changed include setting an Astrometry.net API key, choosing how to calculate the radius of the target object, and choosing which VizieR catalog (or SIMBAD) to search for reference stars. Once the user has chosen their settings the program can be run. 28

Fig. 4.2: Abridged version of the steps Photometry+ takes to perform differential photometry.

Autonomous differential photometry follows the same steps as manual differential photometry. These steps are calibration of the telescope image, finding background radiation noise to subtract out, locating reference stars of known magnitude in the image, and comparing the reference stars to the target object to calculate a comparative magnitude of the target object. These individually calculated target object magnitudes are averaged together to create the final target object magnitude for that image. An error is also calculated for this magnitude through a user's choice of standard deviation, weighted magnitude, using error stars of known magnitude, or a jackknife method for photometric uncertainties (Anderson, et. al., 2000). This process is repeated for many images taken at different points in time. After calculating the magnitudes for every image in a set, Photometry+ generates a publication-quality light curve (like the light curve in Figure 4.3, a graph of the magnitude of a star over time). 29

Fig. 4.3: Light curve made with Photometry+ utilizing GBO data (top) matched with data from AAVSO, where the Photometry+ results are green and AAVSO results are blue (bottom). The match in the bottom figure demonstrates that Photometry+ works with an accuracy comparable to other top photometric pipelines. Figure generated by Ava Covington.

4.3.1 User Interface Design

The user interface for Photometry+ was designed with open source PyQt5 (PyQt, 2021). The main components of the user interface include a page to create a new project, a "My Projects" page where users can view their already created projects, an "About" page where users can learn more about the program and its creators, a page where users can change their default settings, and a page with frequently asked questions. This user interface was designed with user-guided development, following the style of user testing 30 outlined by Steve Krug (Krug, 2010 and 2014). Two development user studies (covered in more detail in Section 4.4.3) were conducted with 3 and 4 participants respectively, and feedback from those user studies were used to improve the GUI to better accomplish the HCI goals of the program. The user interface and its uses will be showcased in Section 5.2 of this thesis.

4.4 Development

Photometry+ was created in two main phases, the first being the creation of the back end (which then went on to become the code version of the program) and the second being creation of the GUI (the desktop version of the program). As part of this somewhat segmented creation several different development and validation techniques were used. In this section we will cover those different development strategies, starting with test- driven development that was used for the back end stage of development, followed by iterative development that was used throughout the development. The development user studies and agile development parts of development and validation were used for the GUI development. Overall, the public release version of Photometry+ took over 2000 hours of development and included 6420 lines of code and 8 UI files.

4.4.1 Test-Driven Development

During the creation of the back end code of Photometry+ test-driven development was used. After the requirements discovery outlined in Section 4.1.1, a skeleton of the overall code was created and tests were designed for every part of the predicted code base. Overall, 72 tests were created. These tests were run in a separate Python file importing functions from Photometry+ and tested every function in Photometry+ with both “Happy Path” inputs where the program is used as intended and “Unhappy Path” inputs where incorrect or invalid data is passed into the function. These tests were run at least every week and sometimes more often, as according to NFR01 all releases had to pass all tests before being uploaded to GitHub and released, even in beta stages of development. Table 4.3 is a small subset of the record keeping of these tests, containing the name and ID of the test, its target function, its purpose, the input(s) for the test, the optimal result, and the 31 most recent result of the test. This spreadsheet was updated every time tests were run, and if any tests currently developed for did not pass then the program was not released. All tests are currently successful because Photometry+ is in its public-release version.

Table 4.3: Example subset of tests used for test-driven development along with most recent results. Test Target Name Purpose Test Data Expected Actual Result ID Result

A.N getWCS Astrom To ensure 1. Star data 1. Return 1. New file etry.net the code without new file created, test using WCS with WCS name astrometry. data returned net runs. 2. Star data 2. Return 0 2. Returns 0 without and print and prints WCS that error error cannot be message message if localized checkup Flag is 1

CLP calibrate Calibrat To ensure 1. A set of 1. Return 1. Calibrated ion test files files calibrated hdul returned without without hdul object WCS can WCS data be calibrated. 2. A file 2. Return 0 2. 0 returned where the header has no “END” keyword

CLN calibrate Normal To ensure 1. A set of 1. Return 1. Calibrated calibrati normal files files with calibrated hdul returned on test can be WCS data hdul object calibrated without stopping the program from running.

RAD find Radius To ensure 1. Regular 1. Radius 1. Radius Radius test the radius image and does not star run out of bound or break the program

CNT star Countin To ensure it 1. Regular 1. Counts 1. Counts Count g test can tally image, counts in an radius, and area center without breaking.

WC world WCS To ensure 1. Regular 1. WCS 1. WCS S Coordin Extracti the WCS image object object ate on test can be System grabbed without breaking

LOC find Locatin To ensure 1. Regular 1. Star 1. Star object Other g test that image and object array array Stars SIMBAD middle can be coordinate reached and star objects can be made

PRF print Print To ensure 1. Array of 1. Print file 1. Print file is Referenc referenc reference star objects is made made eToFile e test stars can be printed without breaking the program

RRF read Read To ensure 1. Name of 1. Array of 1. Array of From referenc reference a file with Star objects Star objects File e test star can be reference read in stars from a file 33

without breaking the program

MA calculate Magnit To ensure 1. Full 1. Program 1. Program G Magnitu ude test magnitude regular does not did not break de and error data break And can be Error calculated without breaking the program

FIL runFiles Multipl To make 1. 1. Array of 1. Array of e sure that Directories Photometry Photometry Photom both kinds containing objects objects etry test of files can a variety of run files

PRS print Print To make 1. Array of 1. Results 1. Results file Results results sure results Photometry file printed printed ToFile test can be objects printed

RCS reduced Reduce To make 1. Array of 1. 0 (or 1. 0 Chi d chi sure the Photometry other Squared squared reduced chi objects number) test squared can be calculated without breaking the program for a set of photometry results.

4.4.2 Iterative Development

To develop Photometry+ we used an iterative development scheme, shown in Figure 4.4, that involved frequent check-ins with stakeholders interested in Photometry+. While iterative development was used in both main stages of development, it worked slightly differently during the coding phase and during the GUI phase. Creating the code version (back end) of the program utilized test-driven development. It started with the requirements discovery outlined in Section 4.1.1, and then launched into designing the system and creating test cases for it. Once the system was designed and test cases created, the system was implemented with documentation happening simultaneously with implementation. The system was then validated on the test cases and deployed. The deployed program was then shown to the stakeholders, who gave more requirements and modified existing requirements. This loop continued until the back end of the program was complete and the public-release code version was ready.

Fig. 4.4: A diagram showcasing the iterative development stages used for all parts of coding Photometry+.

After the public-release code version was ready we moved on to creating the desktop version of Photometry+. Design and prototyping for Photometry+ happened only once, but was very thorough, with examples in Figure 4.5. Three different designs for the user 35 interface were tested with a group of stakeholders, two designed with a desktop application in mind in different color schemes and one designed for a website. After testing the prototypes with stakeholders there was a preference for the desktop version in a silver color scheme. Once the prototype was finished and clearly defined we went into development for it. This development involved implementation and modifying the GUI, validating and testing the GUI manually, and then using development user studies as outlined in Section 4.4.3 to improve it. This section of the program was also regularly deployed as part of the Agile development used on it that will be outlined in Section 4.4.4.

Fig. 4.5: Three example screens of Photometry+ prototypes, with the desktop versions on top and the website version on the bottom. 36

4.4.3 Development User Studies

Jacob Nielson’s research into the relationship between user studies done and usability problems found shows that the relation of usability problems to user studies is approximately defined by the following equation (Nielson, 2000):

U = N (1-(1- L ) n ) (4.1)

Where U is the usability problems discovered, N is the amount of usability problems in the program, and n is the number of users tested. L represents the percentage of usability problems found by one user, which through testing Nielson determined to be about 31%. By this math, user studies with only one user can discover 31% of the problems, user studies with two participants can discover over half, and using five participants can find about 90% of usability problems with a program.

It is with this principle in mind that we conducted two smaller user studies in order to locate and solve usability problems in the program. The first user study was conducted with three participants and can be estimated to have found about 67% of the usability problems with the program that were then fixed or improved. In order to continue to improve the program, we performed a second user study with four participants on the version of the program with all usability issues from the first user study solved. Four participants, according to Nielson’s formula, would find 77% of the usability problems with the program, however there were 42% less usability problems identified in the second user study. This indicates that because four participants will find a higher percentage of usability problems than three participants the number of usability problems did go down because of the development user studies.

4.4.4 Agile Development

We implemented Agile development style into the creation of our GUI based on the agile manifesto (Beck, et. al., 2001), as well as following the principles of scrum (Scrum Principles, 2017). In using this development, much of the GUI development for 37

Photometry+ was completed in sprints following the above principles. According to the main statements of the agile manifesto we (i) put the end-user first, (ii) created software that worked well and was released iteratively while adjusting documentation outside of sprints, (iii) worked with the astrophysicists on the Photometry+ team weekly to ensure that Photometry+ fulfilled their requirements, and (iv) allowed flexibility in the plans made for development for the addition of new requirements and changes to existing requirements. In following the Scrum principles, we (i) released new work to GitHub at the end of every sprint to ensure transparency, (ii) remained self-organized and set regular self-chosen deadlines, (iii) collaborated with all members of the team as per the agile manifesto, (iv) completed tasks in order of priority, not in order of ease, (v) boxed time using timers and the Timeular (Timeular, 2021) system, and (vi) checked in frequently with stakeholders to continue iterative development unimpeded.

4.5 Data Design

There are two kinds of output and input CSV files used in Photometry+, files for Photometry object results and files for Star objects. The formatting for each is as follows:

Photometry Results File: File Name,JD,Magnitude,Error, filename.fts,2458894.8388310187,15.65,0.379867, ... X,Reduced Chi Square: 1.20,

Stars File: ID,Right Ascension (Decimal Degrees),Declination (Decimal Degrees),Radius (pixels),Photons,Magnitude,Calculated Target Magnitude,Error, Ref1,119.704825,16.117481,25,43099.0,16.22,0.0,523.0, …

Using these files, users can pass results and star objects back into the program, allowing them to use their own reference stars, create new light curves based on their own data, and in other words increase their ability to customize Photometry+. These files are only 38 used with the code version with user interaction, the desktop version of Photometry+ saves its data in difficult to read formats for faster use with the GUI.

4.6 Documentation

Photometry+ has extensive documentation for both its code and desktop versions, an excerpt of which is viewable in Appendix A. Appendix A also contains information on how to access the full versions of the documentation. Additionally all code used to create Photometry+ is clearly and completely commented for easy use, as shown in the code excerpt included in Appendix B.

5 Scenarios of Use

The two versions of Photometry+, the code version and the desktop version, are both located on GitHub and used in different ways. As noted in Chapter 4, the code version of Photometry+ allows for more flexibility, control, and utility while the desktop version of Photometry+ is easier to use, abstracts the internal workings of the program, and provides teaching benefits beyond the creation of light curves. With that being said, this chapter is split into two main components, with Section 5.1 covering the use of the code version of Photometry+ and Section 5.2 covering the use of the desktop version, including screenshots of the application itself.

5.1 Code Version

The code version of Photometry+ is a set of functions that can be accessed in one of two ways: through making a main Python file that imports photometry.py or from running its functions in the Python terminal. The code version on Github is available both as the raw, newest version of the code as well as carefully curated releases of the most stable version of recent code. This version was created first because the stakeholders queried from Section 4.1.1 specifically requested this version as their most preferred way of accessing Photometry+.

Although this version is not as easy to use as the desktop version, several design decisions were made to attempt to make it easier to use. All of the functions have extensive comments both in and around them documenting their use, their inputs, their outputs, and other important information. Additionally there is a consolePrintFlag setting that turns on comprehensive error and action reporting to the Python terminal, allowing users to see what Photometry+ is doing in real time. Finally, there is an extensive user guide for the code version of Photometry+ available and referenced in Appendix A. 40

5.1.1 Use Cases

Each use case shown in Figure 5.1 represents an individual function that users can call and run in the code version of Photometry+, with some functions calling other functions within them. They are described in more detail in Table 5.1, and given numerical designations such as CUC01, which represents Code version Use Case 1.

Fig. 5.1: Use cases for the code version of Photometry+.

Table 5.1: This table describes the use cases in Figure 5.1 in more detail. Code Version Use Case Descriptions

CUC01 calibrateFile Users can upload an uncalibrated file with the files they need for calibration and get one calibrated file back.

CUC02 findBlankSkyRadiation Users can find the median number of counts in “blank” sky pixels not containing stars, representing the cosmic background radiation in that image.

CUC03 findRadius Users can find the radius of a star or find the full- width half-maximum of the image to use as a radius. 41

CUC04 starCount Users can find the number of counts within the area of a star minus the background radiation.

CUC05 findOtherStars Users can autonomously find other reference stars in the vicinity of the target object for comparison using VizieR catalogs or SIMBAD. These stars already have their magnitudes attached.

CUC06 printReferenceStarsToFile Users can print a copy of the reference stars used to a file.

CUC07 readReferenceStarsFromFile Users can read out reference stars from a file into usable data.

CUC08 printResultsToFile Users can print the results of photometry to a file.

CUC09 plotResults Users can create a light curve based on the results of photometry.

CUC10 calculateMagnitudeAndError Users can calculate the magnitude of a target object using reference star magnitudes, as well as the error of that calculation

CUC11 removeReferenceOutliers Users can specify a z-score to use when automatically discounting outlier reference stars from the calculation.

CUC12 addWCS Users can add WCS to a file that does not have a WCS attached to it using the Astrometry.net API.

CUC13 runPhotometry Users can run photometry from start to finish, putting in any number of files and receiving a magnitude and error.

CUC14 createCMD Users can create a CMD diagram based on the stars in an image.

CUC15 changeSettings Users can change the settings of the program from the defaults in order to get better fitting results.

CUC16 getSettings Users can get the current settings of the program.

2 CUC17 calculateReducedChiSquared Users can calculate the x r value of a set of data. 42

5.2 Desktop Version

The desktop version of Photometry+ has much of the same functionalities as the code version, as shown in Section 4.1.2, however they are more abstracted from the user. While each of the functions described in Section 5.1.1 are still a part of the desktop version, users can no longer call on those functions individually. They are instead called by the program as part of creating a photometry project. The GUI version of Photometry+ was chosen to be a desktop application rather than a website because stakeholders preferred having a downloadable version they could keep themselves rather than having a website that is on a server and could be taken down. This section will also display screenshots of the program and describe what is occurring in them.

5.2.1 Use Cases

The use cases in Figure 5.2 describe concrete actions users can take with Photometry+ (with the exception of learnPhotometry, which is a side benefit of taking other actions). These use cases are described in more detail in Table 5.2 as well as given a numerical designation, such as Desktop version Use Case 1 (DUC01).

Fig. 5.2: Use cases for the desktop version of Photometry+. 43

Table 5.2: Detailed descriptions of the use cases described in Figure 5.2. Use Case Descriptions

DUC01 changeDefaultSettings Users can change the default settings for the program and have those settings saved for the next time they open the program.

DUC02 changeProjectSettings Users can change the settings for a specific project without changing the default settings.

DUC03 newProject Users can create and run a new project, which results in a result page with a light curve and information on the results.

DUC04 viewAboutInfo Users can view a page containing information about the program and program’s development team.

DUC05 viewFrequentlyAskedQuestions Users can view a page with frequently asked questions about the program.

DUC06 walkthroughMode Users can turn on a walkthrough mode that steps through the program slower for better understanding and gives more control over the program.

DUC07 downloadLightCurve Users can download a result light curve.

DUC08 learnPhotometry Users can use the program to learn more about differential photometry.

DUC09 downloadCsv Users can download a result csv file.

DUC10 viewMyProjects Users can view a collection of all saved projects.

DUC11 viewSpecificProject Users can view the results of a specific project.

DUC12 modifyReferenceStars Users can choose which reference stars to use in their calculations. 44

5.2.2 Home

The first screen the user sees when opening the desktop version of Photometry+ is the “Home” screen, which allows users to navigate to all other sections of the program and start a new project. Figure 5.3 shows the home screen with navigational tabs at the top and the new project button prominently displayed under the logo.

Fig. 5.3: The Home screen of the desktop version of Photometry+.

5.2.3 New Project

The “New Project” page, shown in its unfilled state in Figure 5.4 and its filled out state in Figure 5.5, is where the user can start a Photometry+ project. On this page the user specifies the name of the project, the target object in the image, and the file containing the image itself. In addition to those mandatory items the user can also choose a variety of optional settings and specify calibration files for their project.

Fig. 5.4: The New Project page of Photometry+ before any user input.

Fig. 5.5: The New Project page of Photometry+ after being filled out by the user.

In order to assist the user with understanding the settings options available to them, each setting, required input, and button comes with a tooltip (like in Figure 5.6) explaining more about that setting or input. These tooltips are activated by hovering the mouse over the button or information symbol for the setting. Additionally clicking on the information button will bring up a pop up window with more detailed information, as shown in Figure 5.7.

Finally, in order to prevent user errors coming from a misclick of the “Submit Project” button, when the submit button is clicked the user is prompted to confirm they would like to continue with these settings. This prompt is shown in Figure 5.8.

Fig. 5.6: Demonstration of the tooltip associated with the catalog setting.

Fig. 5.7: Demonstration of the pop up window associated with clicking on the information button next to the catalog setting.

Fig. 5.8: Confirmation prompt after user tries to submit their project. 48

5.2.4 Processing and Walkthrough Mode

After the user submits their project the program will begin processing the images. This can take some time depending on the type and quantity of images, so the processing screen is built with a loading bar that updates as the program processes and text that informs the user of what is currently happening. The image currently being worked on is also displayed in the left side of the screen and is labeled with its file name so the user knows what file is currently active. This has a side benefit of allowing the user to inspect the quality of their uploaded telescope images while they are being processed. This stage of the processing is shown in Figure 5.9.

One of the settings for Photometry+ dealing with this processing stage is “Walkthrough Mode”, which allows the user to inspect certain stages of the process. The most notable addition of walkthrough mode is allowing the user to inspect the reference stars chosen by the program. During processing Photometry+ will either find or upload reference stars based on the users’ preference and then circle both the target object and the chosen reference stars on the telescope image displayed on the left. Then, if walkthrough mode is on, the user is presented with a list of the reference stars, their magnitudes, and the magnitude that the reference star calculates for the target object. From there users can check or uncheck these reference stars based on these factors and have them excluded from the final magnitude calculation. The start of this phase of walkthrough mode is shown in Figure 5.10 with all stars chosen initially checked. In Figure 5.11 reference stars Ref6 and Ref20 are unchecked, and they are no longer circled in the lower right hand side of the target image. This function is important to allow the user to easily visually inspect the image and the reference stars for stars that are too bright, too dim, or too close to other stars (though Photometry+ does automatically detect when stars are too close to them and excludes them from the calculation automatically). Once the user has confirmed their choice of reference stars, they are given the option to continue to use walkthrough mode to step through every image they have uploaded or to turn off walkthrough mode and use their selected reference stars for the rest of the images automatically. 49

Fig. 5.9: Initial processing screen after a new project is submitted.

Fig. 5.10: Walkthrough mode reference star selection with all reference stars checked. 50

Fig. 5.11: Walkthrough mode reference star selection with Ref6 and Ref20 unchecked.

Fig. 5.12: Processing screen after reference stars are confirmed. 51

No matter what the user picks, the program will move on to continue processing other images. While processing the other images after the initial batch of reference stars are located or chosen, these reference stars will be highlighted on the following images while processing, as shown in Figure 5.12.

5.2.5 Project Results

After a project is processed and saved the user will gain access to a project results page, shown in Figure 5.13, where they can view and download that project’s light curve and 2 results CSV file, average magnitude of the target object, average measurement error, � r value, and a list of the files discarded. This page also has information icons that work as shown in Section 5.2.3 that describe what these values mean and the reasons why files can be discarded.

Fig. 5.13: Project results page for a DO Dra project. 52

5.2.6 My Projects

All previously created projects are saved for users to view. On the “My Projects” page, shown in Figure 5.14, users can view and manage their created projects. Selecting a “View Project” button results in the user being taken to the page described in Section 5.2.5 for that project. This screen also displays a snapshot of the information for the project so the user can keep track of them without having to view their page.

Fig. 5.14: Project management page for Photometry+.

5.2.7 About

The “About” page of Photometry+ provides information about Photometry+, links to the user guides for Photometry+, information on the development team, and how to cite Photometry+ in publications, projects, and presentations. This is where users can go to get more information about the program and is shown in Figure 5.15.

Fig. 5.15: The About page of Photometry+, broken into three of the scrollable sections in order.

5.2.8 Settings

The “Settings” page for Photometry+, shown in Figure 5.16, is where users can set their most commonly used settings to a new default value. This new default value will automatically populate into any new projects they start and allow them to save the time they otherwise would have spent filling in their (for example) Astrometry.net API key 54 every time they started a new project. These default settings can be changed for individual projects, but this is the starting default.

Fig. 5.16: The default settings page of Photometry+.

5.2.9 FAQ

The frequently asked questions page of Photometry+, shown in Figure 5.17, has a collection of helpful answers to common questions (the list of which is currently short) and information needed to contact the developer. This page will be expanded as more users join Photometry+.

Fig. 5.17: Frequently asked questions page of Photometry+.

6 Practical Astronomy Applications

As a part of the iterative development outlined in Section 4.4.2, Photometry+ was frequently released and used throughout its development. Because of this iterative release schedule, many of these projects were completed at different stages of development of Photometry+. Many of the projects below contributed to the development of the requirements for Photometry+ as well as being a part of the thorough testing the program underwent. In this section we will outline the different astronomy projects Photometry+ has been a part of already, is currently a part of, or will be a part of in the future.

6.1 Light Curve Analysis of Transient Nova AT2019tlu

AT2019tlu was discovered on October 28th, 2019 (Tomasella et. al., 2019) with a discovery magnitude of 15.4 in the Johnson V filter (AT2019tlu, n.d.). The GBO was used to observe the transient nova starting on October 31st (Tudor and Huerta, 2020). At the time in which this project was conceived, Photometry+ did not exist. The first usable version of Photometry+ was therefore developed for this nova project. This very basic version of Photometry+ had no UI and relied on hand-measurements for several calculations (for example, the blank sky counts were measured by hand and included in the program). However, it performed with acceptable accuracy for the scale of the project and was an excellent starting point.

Photometry was performed on the star using the observatory’s B, V, R, and Luminance filters for exposure times of 5, 10, 30, 60 or 300 seconds. The plan was to take data for about a month, however this turned out to be impossible as the star faded out of the sight of the observatory telescope on November 8th. Additionally, due to a telescope tracking error, the observations from November 3rd to November 6th were not able to be used. Nonetheless, the peak of the nova, shown in Figure 6.1, and the fading rate of the nova were captured in the usable data.

Fig. 6.1: This photometry data shows the star AT2019tlu. This image is made of the data from the B, V, and R filters of the Great Basin Observatory, and AT2019tlu is circled.

There were five nights of observations in the B, V, and R filters on October 31st, November 1st, November 2nd, November 7th, and November 8th. The measurement of V = 15.4 on October 27th was the discovery magnitude reported on Astronomer’s Telegram in an early detection (Tomasella et. al., 2019). The light curve from the GBO is shown in Figure 6.2. We caught the peak with an obvious jump from 15.2 mag to 14.3 mag on November 1st followed by the beginning of the descent to 14.36 mag (a lower magnitude corresponds to a brighter object). At 14.29 mag at the peak, AT2019tlu was slightly dimmer than Pluto is from Earth.

Fig. 6.2: The light curve of AT2019tlu shows the peak of the nova and its rate of decline in the V filter. The first data point is from the Astronomer’s Telegram, the other five points are from the GBO.

6.2 Monitoring For Rare Low-Flux States of Intermediate Polars

As part of the intermediate polar monitoring project discussed in Section 2.2.2 we have been monitoring the flux of over ten IPs and LLIPs. An example of the LLIPs being monitored is DW Cnc, imaged in Figure 6.3 along with the reference stars used, in the V magnitude for one and a half months. The magnitude hovered in between 15 and 16 mag, dipping slightly lower than 16 mag for two days. The error bars are calculated using standard deviation from the magnitudes calculated for each reference star during the photometry process. At the time this project was started, Photometry+ was still in beta development, however over the months this project has run, several versions of Photometry+ were used. As of the writing of this thesis, this project is using the public- release code version of Photometry+.

Fig. 6.3: Calibrated image data from February 15th. DW Cnc is circled in green, and the reference stars used for photometry are circled in white.

For three days in the middle of observation there was a malfunction with the GBO which prevented the capture of useful data. Additionally some days data could not be collected by the GBO, possibly due to bad weather or that our observations did not make it into that night’s observing queue. But overall the light curve, shown in Figure 6.4, is a good representation of DW Cnc’s activity over the observing period.

Fig. 6.4: The light curve of DW Cnc showing magnitude over a period of a month and a half in the Johnson V filter.

The magnitude of the star remained stable throughout observation, as expected for a star not currently undergoing some type of variable activity. DW Cnc has not gone into a low state in the time it was being observed by the GBO, although it has been shown to go into a low state previously, making it an excellent target (Segura Montero et. al. 2020). Shown in the light curve above, the standard deviation of the light curve data is somewhat high (0.2 to 0.4), likely due to DW Cnc’s proximity to the horizon during monitoring. In order to quantify that the star is not currently going through a variable state a common measure is to calculate the reduced χ2 value of the light curve using equation (6.1).

∑ (6.1)

The reduced χ2 value formula finds the variation in the magnitude measured each night

(mi) and the mean magnitude ( ) and then normalizes it by dividing over the error for the 61

night ( ) and squares that number to prevent negative values. Then the whole calculation is divided over N-1, where N is the amount of nights of measurements. This value is approximately 0.13 for the light curve of DW Cnc. The astronomy community widely considers 3 the threshold for meaningful variability, and so this value quantifiably shows the lack of variability in the star during this observing period. However, this value being lower than one indicates that Photometry+ may have overestimated the uncertainties on individual measurements. As this calculation was done on an early version of Photometry+, this information was used to improve the error calculating of Photometry+.

While a relatively unchanging light curve may not be the most exciting result, this light curve was a proof of concept for autonomous photometry. Being able to monitor the magnitude of a star in real time will be critical when it comes time to trigger an X-Ray observation of an IP or LLIP as mentioned in Section 2.2.2. The baseline magnitude of DW Cnc is now known to be in the 15 to 16 mag range, so if it is detected at around 17 or 18 mag it will be an easy decision to take the X-Ray observation. This monitoring project is ongoing at the time of the creation of this document, with many stars being monitored for a dip in flux (Tudor, 2020).

6.3 Teaching Stellar Evolution with a Generated CMD

The CMD, or Color Magnitude Diagram, is a diagram that plots the magnitude of a star against its color, as defined by the difference in magnitude between observations in two different filters. Thus, when a whole cluster of stars is plotted on a CMD it is possible to learn about the age of the cluster and its distance from Earth. A standard method for CMD creation is to perform photometry on the stars in the Johnson V and B filters. On the vertical axis, the V magnitude of the star is plotted and on the horizontal axis the B magnitude with the V magnitude subtracted is plotted, representing the color of the star.

Because of the value of the CMD in learning about stellar evolution, it was used in a lesson plan for an astrophysics measurements course at the University of Nevada, Reno (AST 310). The premise of the lesson was that after learning about photometry, each 62 student would perform V and B filter photometry on 2-3 chosen stars as a homework assignment, and then their values would be used to create a class CMD diagram of the cluster M67. However, while this would be a simple (albeit time consuming) assignment for the students, it was a difficult task for the instructor to facilitate because (i) the instructor needed to perform photometry in both V and B filters for at least the 30 stars the class was going to use and (ii) those stars had to be hand-selected to get a good mix of stars that show the correct shape of the cluster on the CMD. Given the established difficulty of performing photometry by hand the professor turned to Photometry+ to generate the CMD (which is FR35 in Section 4.1.2). Photometry+ was used to generate a CMD based on every star in the image, and then the professor was able to hand select stars that would form the correct overall shape of the cluster (matching the CMD with all stars). Additionally the professor was able to grade the accuracy of the students’ photometry results using the Photometry+ calculations. The CMDs generated for the lesson plan by Photometry+ and generated by the students after the lesson are shown in Figure 6.5.

Fig. 6.5: CMD generated by Photometry+ from images, with stars selected for the lesson plan in red (left) and the CMD generated from values calculated by the students (right).

6.4 Studying the A0620-00 Black Hole X-Ray Binary System

A student, Matthew Neill, is studying the low mass black hole X-Ray binary system A0620-00 (Pictured in Figure 6.6) by using Photometry+ and the GBO to monitor the 63 system1. Neill observes A0620-00 in the V-band, I-band, and i’-band and is working on determining the periodicity of the system based on its flux in order to accurately measure the accretion disk around the black hole of the system.

Fig. 6.6: Image of A0620-00 (circled) in the I-band taken by the GBO.

Measuring the periodicity of any system requires large amounts of data, as one typically needs to sample the entire orbit at least three times. Additionally, these measurements are taken three times with different filters. Photometry+ allows for easy tracking of the objects in a reasonable amount of time, and has led to the creation of the light curve in Figure 6.7.

Fig. 6.7: Light curve of A0620-00. Figure generated by Matthew Neill.

1 See www.youtube.com/watch?v=eGlanCfipzs&feature=youtu.be for a link to a video where Matthew Neill explains this project. 64

6.5 Planned Observing Projects

In addition to the aforementioned studies both completed and in progress using Photometry+, more research is planned that relies on Photometry+. One such future project is a continuation of the project outlined in Section 6.4, the study of A0620-00. Neill is currently studying the orbital period of A0620-00 in order to ascertain the precision with which the GBO can be used to make these kinds of measurements. If the GBO proves precise enough, it can be used to measure the orbital period of other X-Ray binary systems. A0620-00 is an excellent starting point for studying if these measurements can be taken with precision as it is bright, well known, and has good measurements taken of it already.

Additionally, Dr. Richard Plotkin has been monitoring another black hole X-Ray binary, V404 Cygni, during the times in which it is visible. He currently has amassed a large amount of roughly ten minute long observations in multiple filters for every night it has been visible with the goal of monitoring its optical variability. Additionally, he has taken hour long observations in order to detect optical flaring, which should help us learn about how the accretion flow and/or black hole jet change in a short amount of time. At this point he has accumulated enough data that it would be difficult to reduce by hand, and therefore Photometry+ will be used to measure the data efficiently. These two future projects (and those already completed) are only examples that we are already aware of, though the use of Photometry+ can extend far beyond to a great variety of other astronomy uses. Due to the robotic nature of the GBO it is an ideal telescope to obtain rapid observations of new transients, like those mentioned above and more. Photometry+ will be integral for rapid reduction of this data in order to bring magnitudes for these transients to the community (such as through the Astronomer’s Telegram).

7 User Studies

The main experiment described in this thesis involves a user study wherein participants were asked to perform tasks related to differential photometry with the fully developed Photometry+. These tasks were done with data from the GBO telescope and a SIMBAD page, shown in Figure 7.1, containing location information for the star DO Dra.

Fig. 7.1: The example SIMBAD page that was provided to participants of the user study.

7.1 Participants

The recruitment for this user study targeted physics and astrophysics students, faculty, and researchers. Recruitment messages were sent out at the 237th meeting of the American Astronomical Society, to the Great Basin Observatory users committee, to the University of Nevada, Reno (UNR), Department of Physics, the UNR astronomy club, and other local astronomy groups. This targeted recruitment ensured that the participants using Photometry+ were a part of its final target audience in order to accurately test whether astrophysics students and researchers who may not use differential photometry often can perform the process with this tool. Participants filled out a pre-study survey that collected demographic information, including gender, education level, astronomy experience, and photometry experience. 66

7.1.1 Age

75% of the participants were in the 18-24 age range, with a few participants in the 25-34 and 45-54 age ranges, as shown in Figure 7.2. This matched the demographics of those recruited, which mainly targeted college-aged students. As a large part of this user study was testing if students could use Photometry+ as a teaching tool, this grouping of younger astronomers was ideal.

Fig. 7.2: Pie chart representing the age of participants in the user study.

7.1.2 Gender

75% of the participants in this user study were male, which is similar to the demographics in the physics field (Porter and Ivie, 2019), with the remaining 25% being a combination of female and non-binary participants as shown in Figure 7.3. While it would be better to have a more diverse group of user study participants, in fields with a large gender bias such as physics, recruitment of underserved minorities is difficult and this limited our study.

Fig. 7.3: Pie chart representing the gender of participants in the user study.

7.1.3 Education Level

41.7% of the participants in this user study were still pursuing their first undergraduate degree, 33.3% had graduated with an undergraduate degree, and the remaining 25% of participants were either in graduate school or had a graduate degree, as shown in Figure 7.4.

Fig. 7.4: Pie chart representing the education level of participants in the user study. 68

7.1.4 Operating System

In Figure 7.5 it is clear that the most common operating system among participants was Windows. Two-thirds of participants used Windows, and one-third of the participants used MacOS and Linux (split evenly). This posed a minor problem during the user study as the study administrator’s computer was a MacOS computer, and several participants expressed unfamiliarity with the MacOS directory system during the directory portion of the user study tasks.

Fig. 7.5: Pie chart representing the primary operating system of participants in the user study.

7.1.5 Astronomy Experience Level

As expected based on the recruiting we performed, all participants had at least a little astronomy experience, with most participants having at least a moderate amount of astronomy experience (seen in Figure 7.6). 69

Fig. 7.6: Pie chart representing the astronomy experience level of participants in the user study.

7.1.6 Photometry Experience Level

Despite that most participants were at least moderately experienced with astronomy, the participants were much less experienced with photometry. As shown in Figure 7.7, 75% of participants stated they were not so familiar or not familiar with photometry.

Fig. 7.7: Pie chart representing the photometry experience level of participants in the user study. 70

7.2 Apparatus

Due to the COVID-19 pandemic all user studies for this research were conducted remotely via the video and messaging application Zoom (Zoom, 2021). Participants were asked to take remote control over the study administrators computer to take a brief quiz, perform some tasks on the program and then take another brief quiz. The tasks performed with Photometry+ were based entirely on the user interface of the program and no interaction with code or Python was required. Each user study took less than forty-five minutes in total.

7.3 Procedure

To maintain consistency between every study, a script was followed when performing the user studies and all emails sent to participants were the same. This ensured that each participant had the same experience to minimize confounding variables. Every study followed the following procedure: 1. The participant was asked to sign a consent form and recording release. 2. The participant filled out a pre-study survey with a unique participant ID. 3. The participant joined a Zoom call with the study administrator. 4. The participant was given remote access to the testing machine with the quizzes and Photometry+ available. 5. The participant was given a differential photometry quiz that briefly assessed their prior differential photometry knowledge. 6. The premise and purpose of the study were briefly explained. 7. The participant was given the task list for the study and the SIMBAD page mentioned above. 8. The participant performed the tasks on the list using Photometry+. 9. The participant was given the same quiz as they took at the beginning to assess the change in their differential photometry knowledge. 10. The Zoom call was ended. 11. The participant filled out a post-study survey. 71

The pre-study survey participants filled out concerned demographic information such as age, gender, education, and level of astronomy and photometry knowledge. The differential photometry quiz scored users out of 12 based on questions about photometric calibration, differential photometry steps, and other related knowledge. This quiz was given twice. The tasks the participants were asked to perform included creating a new Photometry+ project targeting DO Dra, changing default settings, and observing the results of their project. After the users finished performing tasks and retook the quiz, they were given a post-study survey that asked questions concerning the look and feel of Photometry+, their confidence in its results, and whether they learned more about differential photometry by using it.

7.3.1 Pre-Study Survey

The pre-study survey was administered to the participants before the user study and the Zoom call was started, and mostly dealt with their demographics. This survey is included in its entirety below: 1. Participant ID: 2. Which age group do you belong to? a. 18-24 b. 25-34 c. 35-44 d. 45-54 e. 55-64 f. 65+ g. Prefer not to answer 3. What is your gender? a. Male b. Female c. Non-binary d. Prefer not to answer 4. What is the highest level of education you’ve completed? a. Did not attend school b. Some school c. Completed High School d. Some college e. Undergraduate degree f. Some graduate school 72

g. Graduate degree 5. If you own a personal computer, which operating system do you use? a. MacOS b. Windows c. Linux d. I do not own a personal computer e. Other (please specify) 6. Do you use computer-based tools for any of the following: a. Work b. Research c. Homework 7. How many hours a week do you spend using the computer? a. 0 b. 0 to 10 hours c. 10 to 20 hours d. 20 to 30 hours e. 40+ hours 8. What level of experience do you have with astronomy or astrophysics? a. A great deal b. A lot c. A moderate amount d. A little e. None at all 9. What is your familiarity with differential photometry? a. Extremely familiar b. Very familiar c. Somewhat familiar d. Not so familiar e. Not familiar f. Not at all familiar 10. Do you think you can learn complex processes through a computer program? a. Yes b. No c. I’m not sure

7.3.2 Differential Photometry Quiz

The differential photometry quiz was administered live on the Zoom call two times, once before completing tasks with the program and once after completing tasks with the program. It is included below, with correct answers bolded: 1. Participant ID: 2. Which of the following types of files are used for telescope image calibration? Select all that apply. 73

a. Flat Field b. Light Frame c. Bias Frame d. Dark Frame 3. How is the magnitude of a star calculated in differential photometry? a. Putting the photons counted in the star into the magnitude formula b. Comparing the star to several other reference stars of known magnitude c. Putting the radius of the star into the magnitude formula d. Use the average magnitude of similar sized reference stars 4. Which of the following steps are part of performing differential photometry? Select all that apply. a. Find target star radius b. Count photons in the target star c. Finding reference stars d. Calculate magnitude and error of star 5. Can calibration frames interact with each other? a. Yes b. No 6. Where can magnitudes for stars be found? Select all that apply. a. SIMBAD b. VizieR catalogs c. AST Data d. Nowhere 7. How many files are needed to perform photometry on a fully calibrated image? a. Participant will be prompted to enter a number: 4

7.3.3 Photometry+ Tasks

The tasks that the participants were asked to perform with Photometry+ are included below, in an abridged form: 1. Change the default settings of the program 2. Create a new project named “DO Dra User Study” and fill in information as if you are targeting the star in the SIMBAD page given, DO Dra. 3. Remove any reference stars with a calculated target magnitude greater than 15.00 4. Go to “My Projects” 5. Delete “DO Dra User Study” 6. View “DO Dra October 16th” and discuss your confidence in these results 7. View the “About” page and discuss your thoughts on the page 8. View the “Home” page and discuss overall impressions of the program 74

7.3.4 Post-Study Survey

The post-study survey was administered to the participants after the user study and the Zoom call was ended, and mostly sought to identify their thoughts on Photometry+ itself. This survey is included in its entirety below: 1. Participant ID: 2. Please rate the following statements: a. Photometry+ is easy to use i. Strongly disagree ii. Disagree iii. Neutral iv. Agree v. Strongly agree b. The user interface of Photometry+ is well designed i. Strongly disagree ii. Disagree iii. Neutral iv. Agree v. Strongly agree c. The user interface of Photometry+ is intuitive i. Strongly disagree ii. Disagree iii. Neutral iv. Agree v. Strongly agree d. It is easy to navigate Photometry+ i. Strongly disagree ii. Disagree iii. Neutral iv. Agree v. Strongly agree e. I am in control of how Photometry+ runs i. Strongly disagree ii. Disagree iii. Neutral iv. Agree v. Strongly agree f. I would be confident in the results Photometry+ produced i. Strongly disagree ii. Disagree iii. Neutral iv. Agree v. Strongly agree 75

g. I have learned more about performing differential photometry than I knew before i. Strongly disagree ii. Disagree iii. Neutral iv. Agree v. Strongly agree h. I know the steps of performing differential photometry i. Strongly disagree ii. Disagree iii. Neutral iv. Agree v. Strongly Agree 3. On a scale of one to five (one being poor and five being the best), please rate your experience with Photometry+ a. 1 b. 2 c. 3 d. 4 e. 5 4. Would you use Photometry+ again if you needed to perform differential photometry? a. Yes b. No c. Maybe d. I’m not sure 5. How satisfied are you with: a. The engagement of Photometry+ i. Not at all satisfied ii. Not so satisfied iii. Somewhat satisfied iv. Satisfied v. Very Satisfied b. The visual aesthetic of Photometry+ i. Not at all satisfied ii. Not so satisfied iii. Somewhat satisfied iv. Satisfied v. Very Satisfied c. The reliability of Photometry+ i. Not at all satisfied ii. Not so satisfied iii. Somewhat satisfied iv. Satisfied v. Very Satisfied 76

d. The quality of Photometry+ i. Not at all satisfied ii. Not so satisfied iii. Somewhat satisfied iv. Satisfied v. Very Satisfied 6. What do you like most about Photometry+? 7. What do you like the least about Photometry+? 8. Did you encounter any problems while using Photometry+? 9. Are there any features you’d like to see added to Photometry+? 10. Any final thoughts?

7.4 Experimental Design

Independent variable: Prior use of Photometry+ Name: Photometry+ Exposure Levels: Before use, after use

Dependent variable: Differential photometry knowledge

The independent variable being manipulated was whether participants had been exposed to Photometry+ before or not. Thus, participants were measured on differential photometry knowledge before and after using the program for the first time. Changing the Photometry+ Exposure variable was predicted to produce a change in the dependent variable, differential photometry knowledge, as our hypothesis was that Photometry+ can teach differential photometry. The amount of entry in this experiment is 12 participants x 2 administered quizzes for 24 phases. Thus this is a 12x2 within-subjects design.

8 Results and Discussion

The primary results of the user studies were derived from the post-study survey, the differential photometry quiz, and from watching the user study itself. The main result from watching the user studies themselves is that timing the user study showed that participants took an average of 10.66 minutes to complete their first photometry project, despite never having used the software prior. This satisfies NFR03 from Section 4.1.3 that the average user can complete their first project in less than 30 minutes. In fact, from all 19 total user studies done (seven for development and twelve for research) it is clear that no matter the definition of ‘average user’ this requirement is satisfied, as shown in Figure 8.1. As expected, those with more foundational astronomy knowledge could get started faster, but even the least experienced astronomers completed the setup in an average of 12 minutes.

Fig. 8.1: Time it took for participants to set up their first Photometry+ project by astronomy experience level.

8.1 Results of Post-Study Survey

Using the post-study survey, we collected data from the participants concerning how easy it was to use Photometry+. For statements like "Photometry+ is easy to use", "The user interface of Photometry+ is well designed", "Photometry+ is intuitive", and 78

"Photometry+ is easy to navigate", users consistently rated their agreement with the statement between "Strongly Agree" or "Agree", which were 5 and 4 respectively on our Likert scale. The average participant scores for those statements were 4.58 for "Photometry+ is easy to use" and 4.33 for "The user interface of Photometry+ is well designed", "Photometry+ is intuitive", and "Photometry+ is easy to navigate", as shown in Table 8.1. When asked questions concerning the aesthetics of Photometry+, users rated them 4.33 on average overall. This number comes from averaging each individual participants’ rankings on all parts of question five in the post-study survey, and then averaging that result for every participant together.. Additionally, on the post-study survey, participants were asked to agree or disagree with the statement "Photometry+ taught me more about performing differential photometry" on a Likert scale. The average value of the responses was 4.08 on a scale where five points was the maximum. This correlates closely with the results presented in Section 8.2.

Table 8.1: Results of post-study survey Likert scale questions. In all of the examples below, the scale starts at “Strongly Disagree” at one and ends at “Strongly Agree” at five.

Statement Average Ranking Standard Deviation

Photometry+ is easy to use 4.58 1.00

The user interface of 4.33 0.49 Photometry+ is well designed

Photometry+ is intuitive 4.33 0.89

Photometry+ is easy to 4.33 0.98 navigate

Photometry+ taught me 4.08 0.79 more about performing differential photometry

Overall Aesthetic 4.33 0.57 Assessment

8.2 Analysis of Dependent Variable Derived From Quiz

The primary HCI goal of Photometry+ is to teach students differential photometry in addition to being an analysis tool. To assess this, we administered a short quiz on differential photometry before and after the participants used Photometry+. As shown in Figure 8.2, the mean score on the quiz that participants took after using Photometry+ measured at 81%, which is 15% more than the 66% mean score from before participants used Photometry+. This difference, when analyzed with ANOVA, is statistically significant (F = 7.54, p<.05).

Fig. 8.2: Detailed information on the performance of participants on a differential photometry quiz before and after exposure to Photometry+.

8.3 Discussion

The data gained from the experiments show that Photometry+ achieves its HCI goals of being easy to use and of teaching differential photometry. Through directly measuring use time, we can ascertain that, even with little experience with photometry and no experience with Photometry+, participants could still perform differential photometry in a reasonable time span, with many of the participants expressing the sentiment that they could repeat the process more quickly if they needed to use the software again. By 80 directly measuring improvement in differential photometry knowledge with the quizzes, we conclude that there is a statistically meaningful knowledge boost associated with using Photometry+. It is apparent that both HCI goals were achieved when looking at the directly measured data.

Photometry+ not only accomplished these goals, but also convinced users that these objectives were met. In addition to being able to use the program in about ten minutes, participants directly ranked Photometry+ as being easy to use and intuitive in the post- study survey. Likewise, participants on average agreed with the statement that they learned more about differential photometry from using Photometry+, which means that users are aware of the learning potential of Photometry+. Participants both measurably increasing their learning and feeling that they learned more effectively is a great endorsement of the power of software to support teaching and performing complex scientific tasks.

9 Conclusions and Future Work

Photometry+ is the combination of UX design and development techniques, astrophysics knowledge, and HCI research into making an easy-to-use software tool that can double as a teaching tool for complex scientific tasks. Like other photometric pipelines, Photometry+ can produce high-quality light curves customized to the target object while achieving its non-photometric goals.

In this thesis we have outlined several development strategies and techniques that have been used to make Photometry+. It is impossible to know whether all were required to meet the HCI goals of Photometry+, however these techniques all contribute to successful software development. Additionally, the use of user studies both for development and validation allows for user participation in all stages of development, augmenting the efficacy of the suggestions of the stakeholders interested in Photometry+. This also ensures that more than 90% of total usability problems with Photometry+ were located and solved. This focus on usability and improvement has resulted in Photometry+’s wide range of successful astronomy projects and users, and will continue to help ensure success for astronomy and photometry projects going forward.

Additionally, although Photometry+ is now in its final release version, there is a lot of work that can still be done on it. This version of Photometry+ provides a strong framework for the addition of other astronomy software tools to be added to it. One example of further work that can be done on the differential photometry aspect of the software is development and testing for performing differential photometry on extended sources, such as galaxies or nebulae. This would likely require the addition of different shapes of apertures as well as extensive testing on extended source images. However, Photometry+ does not need to be limited to differential photometry on different sources. Much like Photometry+ includes the creation of CMDs as side functionality to single- source differential photometry, other graphs and functions can be added to the program. And in adding those functionalities it may be worthwhile to modify the desktop version 82 of Photometry+ to include not just differential photometry projects, but rather projects of all kinds. As astronomers find more functionality they’d like to use in Photometry+, it can be modified and expanded.

The final user studies more than validated the accomplishment of the HCI goals of making Photometry+ easy to use and able to teach photometry. This addresses a clear need for scientific software designed with end-users in mind, and can decrease the difficulty involved with learning new scientific methods. The methodology detailed in this thesis could be used to create software for any variety of scientific tasks in a broad variety of fields, and our results show that it works for scientists of all levels of experience. We highly recommend that authors of scientific software of all disciplines try taking the development and research in this thesis for addition to their work, in the hope of expanding UX design and HCI in all fields of science in order to create more gentle learning curves for beginning scientists everywhere. Photometry+ is an open source program that will continue to be worked on, expanded and adjusted. Its ability to create accurate, high-quality light curves and teach students differential photometry makes it an excellent candidate for adoption by other telescopes or projects. 83

Appendix A

The documentation for the code version of Photometry+ is found on a Google doc linked here: https://docs.google.com/document/d/156hhJvwQ5JsuQsuCONkSsc4z2qmjbsZ71uLW4X OGpw8/edit?usp=sharing

A demonstration of the uses of the desktop version of Photometry+ is found here: https://youtu.be/Ibk_Gbu_A6k

Appendix B

The following is an excerpt from the Photometry+ main code file, photometry.py demonstrating the documentation that happens in the code itself. The function displayed here is reducedChiSquared, and the start of the function (and all functions) is a code block displaying the name, returned values, parameters, and purpose of the functions. Furthermore, the code is extensively commented with one line comments in the function.

""" NAME: reducedChiSquared RETURNS: The reduced chi squared value of the data PARAMETERS: An array of Photometry objects (info) PURPOSE: Calculate the reduced chi squared value of a set of results in order to measure variability """ def reducedChiSquared(info): global settings # Make sure an array is passed in try: len(info) except TypeError: info = [info] if len(info) <= 1: # if invalid data is passed in, inform user and exit if settings.consolePrintFlag == 1: print("Reduced Chi Squared error: Reduced chi squared cannot be calculated for", len(info), "value(s).") return 0 # Calculate the average magnitude of all items mBar = 0 85 for i in range(len(info)): mBar = mBar + info[i].magnitude if info[i].error == 0: # if the error values are 0, reduced chi squared cannot be calculated if settings.consolePrintFlag == 1: print("Reduced Chi Squared error: Reduced chi squared calculation needs valid error values.") return 0 mBar = mBar/len(info) # Calculate the reduced chi squared value chi = 0.0 for i in range(len(info)): chi = chi + ((info[i].magnitude - mBar) / info[i].error) ** 2 if info[i].error == 0: # if the error values are 0, reduced chi squared cannot be calculated if settings.consolePrintFlag == 1: print("Reduced Chi Squared error: Reduced chi squared calculation needs valid error values.") return 0 chi = chi / (len(info)-1) return chi

References

Andersen, J. “Accurate Masses and Radii of Normal Stars.” The Astronomy and Astrophysics Review, vol. 3, no. 2, 1991, pp. 91–126., doi:10.1007/bf00873538.

Anderson, W. L., et al. “A Simple and Direct Measure of Photometric Uncertainties.” NASA/ADS, 2000, ui.adsabs.harvard.edu/abs/2000IAPPP..81...16A/abstract.

Andronov, Ivan L., et al. “Rare Low State of DO Dra, the Magnetic Dwarf Nova=Outbursting Intermediate Polar.” ATel, The Astronomer's Telegram, 2017, www.astronomerstelegram.org/?read=10477.

“APASS: The AAVSO Photometric All-Sky Survey.” AAVSO, 2021, www.aavso.org/apass.

The Astropy Collaboration, et al. “Astropy: A Community Python Package for Astronomy.” Astronomy & Astrophysics, vol. 558, 2013, doi:10.1051/0004- 6361/201322068.

The Astropy Collaboration, et al. “The Astropy Project: Building an Open-Science Project and Status of the v2.0 Core Package.” The Astronomical Journal, vol. 156, no. 3, 2018, p. 123., doi:10.3847/1538-3881/aabc4f.

AT 2019tlu. (n.d.). Retrieved from https://wis-tns.weizmann.ac.il/object/2019tlu

Bath, G. T. “Symbiotic Stars - A Binary Model With Super-Critical Accretion.” Monthly Notices of the Royal Astronomical Society, vol. 178, no. 2, 1977, pp. 203–217., doi:10.1093/mnras/178.2.203.

Beck, Kent, et al. Manifesto for Agile Software Development, 2001, agilemanifesto.org/. 87

Britannica, The Editors of Encyclopaedia. "Photometry". Encyclopedia Britannica, 27 Apr. 2017, https://www.britannica.com/science/photometry-astronomy. Accessed 17 February 2021.

Chandrasekhar, S. “The Maximum Mass of Ideal White Dwarfs.” The Astrophysical Journal, vol. 74, 1931, p. 81., doi:10.1086/143324.

Chandrasekhar, S., and E. A. Milne. “The Highly Collapsed Configurations of a Stellar Mass.” Monthly Notices of the Royal Astronomical Society, vol. 91, no. 5, 1931, pp. 456–466., doi:10.1093/mnras/91.5.456.

Chiozzi, G., et al. “TRENDS IN SOFTWARE FOR LARGE ASTRONOMY PROJECTS.” ESO, 2007, www.eso.org/~gchiozzi/MyDocs/ICALEPCS2007/ICALEPCS2007-MOAB03.pdf.

Cross, N., Collins, R., Read, M. et al. “A Matched Aperture Photometry Pipeline Incorporated into the WFAU Archives” Astronomical Data Analysis Software and Systems XXIII., 485, 2010, p.371

Deeg H.J., and Alonso Roi. Transit Photometry as an Exoplanet Discovery Method. In: Deeg H., Belmonte J. (eds) Handbook of Exoplanets. 2018. Springer, Cham. https://doi.org/10.1007/978-3-319-30648-3_117-1

Fausett, Jacob et al. "Automated Pipeline for the Continuous Monitoring of KIC 8462852 Using the Great Basin Observatory." American Astronomical Society Meeting Abstracts \#234. 2019.

Ferrero, A., et al. “The Photometry Pipeline of the Watcher Robotic Telescope.” Advances in Astronomy, vol. 2010, 2010, pp. 1–5., doi:10.1155/2010/715237. 88

GBNP Foundation. “Great Basin Observatory.” Great Basin Observatory | Dark Skies...Bright Minds, 2020, greatbasinobservatory.org/.

Güçsav, Bülent B., et al. “A Pipeline for the ROTSE–IIId Archival Data.” Experimental Astronomy, vol. 33, no. 1, 2012, pp. 197–209., doi:10.1007/s10686-011-9283-9.

Hinton, Samuel, and Dillon Brout. “Pippin: A Pipeline for Supernova Cosmology.” Journal of Open Source Software, vol. 5, no. 47, 2020, p. 2122., doi:10.21105/joss.02122.

Huang, Weirong, et al. “Robust Automated Photometry Pipeline for Blurred Images.” Publications of the Astronomical Society of the Pacific, vol. 132, no. 1013, 19 May 2020.

Ivezić, Željko. “Mapping the Milky Way with SDSS, Gaia and LSST.” Proceedings of the International Astronomical Union, vol. 5, no. H15, 2009, pp. 188–189., doi:10.1017/s1743921310008677.

“Interactive Systems.” Encyclopedia.com, Encyclopedia.com, 15 Feb. 2021, www.encyclopedia.com/computing/news-wires-white-papers-and-books/interactive- systems#:~:text=Interactive%20systems%20are%20computer%20systems,examples%20 of%20graphical%20interactive%20systems.&text=Games%20and%20simulations%20ar e%20interactive%20systems.

Kennedy, M. R., et al. “X-Ray Observations of FO Aqr during the 2016 Low State.” Monthly Notices of the Royal Astronomical Society, vol. 469, no. 1, 2017, pp. 956–967., doi:10.1093/mnras/stx880.

Kochanek, C. S., et al. “The All-Sky Automated Survey for Supernovae (ASAS-SN) Light Curve Server v1.0.” Publications of the Astronomical Society of the Pacific, vol. 129, no. 980, 2017, p. 104502., doi:10.1088/1538-3873/aa80d9.

Kohler, Susanna. “A Black Hole X-Ray Binary Rises.” AAS Nova, 14 Nov. 2018, aasnova.org/2018/11/14/a-black-hole-x-ray-binary-rises/.

Krug, Steve. Rocket Surgery Made Easy: the Do-It-Yourself Guide to Finding and Fixing Usability Problems. New Riders, 2010.

Krug, Steve. Don't Make Me Think, Revisited: a Common Sense Approach to Web Usability. New Riders, 2014.

Landau, Elizabeth. “What the Obsolete Art of Mapping the Skies on Glass Plates Can Still Teach Us.” Smithsonian.com, Smithsonian Institution, 15 Apr. 2019, www.smithsonianmag.com/science-nature/obsolete-art-mapping-skies-glass-plates-can- still-teach-us- 180971890/#:~:text=These%20photographic%20plates%20represent%20the,like%20film %20in%20a%20darkroom.&text=Historic%20plates%20aren't%20just%20relics.

Lang, Dustin, et al. "Astrometry.Net: Blind Astrometric Calibration of Arbitrary Astronomical Images." The Astronomical Journal, vol. 139, no. 5, 2010;2009;, pp. 1782- 1800.

Macaulay, Catriona, et al. “Usability and User-Centered Design in Scientific Software Development.” IEEE Software, vol. 26, no. 1, 2009, pp. 96–102., doi:10.1109/ms.2009.27.

Madore, Barry F., and Wendy L. Freedman. “The Cepheid Distance Scale.” Publications of the Astronomical Society of the Pacific, vol. 103, 1991, p. 933., doi:10.1086/132911. 90

Mommert, M. “PHOTOMETRYPIPELINE: An Automated Pipeline for Calibrated Photometry.” Astronomy and Computing, vol. 18, 2017, pp. 47–53., doi:10.1016/j.ascom.2016.11.002.

Montero, O Segura, S H Ramírez, and J Echevarría. “New Observations of DW Cnc : Where Is the 38 Min Signal?” Monthly Notices of the Royal Astronomical Society, August 2020. https://doi.org/10.1093/mnras/staa856.

Nielson, Jakob. “Why You Only Need to Test with 5 Users.” Nielsen Norman Group, 2000, www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/.

Ochsenbein, F., et al. “The VizieR Database of Astronomical Catalogues.” Astronomy and Astrophysics Supplement Series, vol. 143, no. 1, 2000, pp. 23–32., doi:10.1051/aas:2000169.

Patel, C., et al. “PALFA Single-Pulse Pipeline: New Pulsars, Rotating Radio Transients, and a Candidate Fast Radio Burst.” The Astrophysical Journal, vol. 869, no. 2, 2018, p. 181., doi:10.3847/1538-4357/aaee65.

Photutils, 2021, photutils.readthedocs.io/en/stable/index.html.

Pogson, N. “Magnitudes of Thirty-Six of the Minor Planets for the First Day of Each Month of the Year 1857.” Monthly Notices of the Royal Astronomical Society, vol. 17, no. 1, 1856, pp. 12–15., doi:10.1093/mnras/17.1.12.

Porter, Anne M., and Rachel Ivie. “Women in Physics and Astronomy, 2019.” American Institute of Physics, 19 Apr. 2019, www.aip.org/statistics/reports/women-physics-and- astronomy-2019.

“PyQt.” Riverbank Computing | Introduction, 2021, www.riverbankcomputing.com/software/pyqt/.

Rampersad, Laurisha, et al. “Improving the Usability of Scientific Software with Participatory Design.” Proceedings of the South African Institute of Computer Scientists and Information Technologists on - SAICSIT '17, 2017, doi:10.1145/3129416.3129899.

Riess, Adam G., et al. “Observational Evidence from Supernovae for an Accelerating Universe and a Cosmological Constant.” The Astronomical Journal, vol. 116, no. 3, 1998, pp. 1009–1038., doi:10.1086/300499.

“Scrum Principles.” SCRUMstudy, 2017, www.scrumstudy.com/whyscrum/scrum- principles.

Shen, K. J., Kasen, D., Miles, B. J., & Townsley, D. M. (2018). Sub-Chandrasekhar-mass White Dwarf Detonations Revisited. The Astrophysical Journal,854(1), 52. doi:10.3847/1538-4357/aaa8de

Shuttleworth, Martyn. “Mesopotamian Astronomy - Babylonian and Persian History.” Explorable, 2021, explorable.com/mesopotamian- astronomy#:~:text=From%201800%20BC%2C%20they%20meticulously,astrologers%2 0as%20much%20as%20astronomers.

Stetson, Peter B. “DAOPHOT - A Computer Program for Crowded-Field Stellar Photometry.” Publications of the Astronomical Society of the Pacific, vol. 99, 1987, p. 191., doi:10.1086/131977.

Timeular, 17 Feb. 2021, timeular.com/.

Tomasella, L., & Benetti, S. (2019, October 28). The Astronomer's Telegram. Retrieved from http://www.astronomerstelegram.org/?read=13232

Tudor, Alexis, and Noah Huerta. “Semi-Autonomous Light Curve Analysis of Transient Nova AT2019tlu.” Nevada State Undergraduate Research Journal (NSURJ), vol. 6, no. 1, 2020, nsurj.com/the-journal/.

Tudor, A. R. Monitoring Visual Accretion to Detect Rare Low-Flux States of Intermediate Polars [Undergraduate Thesis]. Reno: University of Nevada, Reno; 2020.

Tudor, A.R., Plotkin, R.M., Shaw, A.W., Covington, A.E., Dascalu, S. (2021, January). User-Guided Development of a Photometric Pipeline for the Great Basin Observatory Robotic Telescope. Poster session presented at the 237th Meeting of the American Astronomical Society.

Tudor, Alexis R. “Photometry+.” GitHub, 2021, github.com/Alexandara/photometry- plus.

Ustyugov, V. A., et al. “The Influence of the Inclination of the Accretor’s Magnetic Axis on the Accretion-Disk Structure in Intermediate Polars.” Astronomy Reports, vol. 57, no. 11, 2013, pp. 811–817., doi:10.1134/s1063772913110061.

Vogt, F.P.A. “Fcmaker: Automating the Creation of ESO-Compliant Finding Charts for Observing Blocks on p2.” Astronomy and Computing, vol. 25, 2018, pp. 81–88., doi:10.1016/j.ascom.2018.08.007.

Wenger, M., et al. “The SIMBAD Astronomical Database.” Astronomy and Astrophysics Supplement Series, vol. 143, no. 1, 2000, pp. 9–22., doi:10.1051/aas:2000332.

“Xerox Star 8010 Information System, 1981.” The Interface Experience: Bard Graduate Center, interface-experience.org/objects/xerox-star-8010-information-system/.

Zhang, Yanxia, and Yongheng Zhao. “Astronomy in the Big Data Era.” Data Science Journal, vol. 14, 2015, p. 11., doi:10.5334/dsj-2015-011.

“Zoom.” Zoom Video, 2021, zoom.us/.