ECEC C RESCIENCE arXiv:2008.07380v1 [cs.CY]10.5281/zenodo.3956058 23 Jul 2020 ordHinsen Konrad ired Buyl de Pierre 5My2020 May 05 4Jl 2020 July 14 eiwdby Reviewed Published dtdby Edited Received DOI ID ID h uhr aedcae htn optn neet exist interests competing no that declared have authors The 1 –90,Lo,France Lyon, F–69007, 710Trń oad– Poland Toruń, 87-100 [ Astronomy / Reproduction ecec C ReScience at available is Data Roukema F. Boudewijn to Com addressed Creative be a should under Correspondence released Roukema, B.F. 2020 © Copyright pnpe eiwi vial at available is review peer Open at available is Code odwj .Roukema F. Boudewijn estimates’ parameter nttt fAtooy aut fPyis srnm n I and Astronomy Physics, of Faculty Astronomy, of Institute ¬ 1 p erdcblt f’onaéddchda space dodecahedral ’Poincaré of Reproducibility Rp] h erdcinsesaedfie omlyi free-licens a in formally defined are steps reproduction The hspprsuisterpouiiiyo h anobservat main the of reproducibility the studies paper This .,dslydi is ,4 n ie ueial nTable in numerically given and 5 4, 3, Figs. in displayed 4.2, iesdsaei h edo omctplg,i particul in topology, cosmic of field the in soware licensed beoln ihhg-ult ouetto,oe ra e an aer oen documentation, high-quality cosmolog with in online data able observational While input 25]. and 7, soware reproduce[14, under free-licensed provide code that source those and even sets but data empirical full fields other the and providing astronomy[3] in papers many are only not that ie nanwgtrpstr ae 8746,floigthe following 0807.4260, named [23] repository RBG08. git new incomplet this a of in steps bined the avail reproduce to public script a and and appeared free-licensing soware they (ii) as and reproducible as availability not t data are short any RBG08 on c of justified results the be the of to content great the too in being int risked g77, soware, compiler, with the compiled of originally heart and the end, at code w 77 It Fortran paper. the current upgrading the in qualitatively and “0807.4260” age owr akg eal rsaelicences. published soware are or authors details other package soware by papers topology cosmic resu lated analysis observational the and RBG08, provi of comments u 3.2 that with Section those are in reproduced me, Po be by a should that developed is results The was section thors. code spatial The the that space[17]. hypothesis spatial working the the of domain under fundamental the of orientation ferred cross-corre optimal co-authors surface-of-last-scattering and the myself used by published paper research topology g bevtoa ae ulse n20 n aifigbo satisfying and 2008 in repr to published attempts paper paper This observational later? ogy decade a author same the i by reproducing for soware free-licensed providing (ii) and Introduction . #1 okm 2020 Roukema – (#11) 6.1 h esnfratmtn n ouetn h reproducibil the documenting and attempting for reason The sasinicrsac ae ae n()pbi,oln obs online public, (i) on based paper research scientific a Is https://lambda.gsfc.nasa.gov/data/map/dr3/dfp/wmap_ https://codeberg.org/boud/0807.4260 2 nvLo,Esd yn nvLo1 NS eted Recherche de Centre CNRS, Lyon1, Univ Lyon, de Ens Lyon, Univ 1,2, https://github.com/ReScience/submissions/issues/41 ID SWH – . . osAtiuin40Itrainllicense. International 4.0 Attribution mons ( Abstract odastro.uni.torun.pl boud swh:1:dir:4f1fe8cf5a01bb4637e31ea938eaa5bc25a2b87b frais ioasCpriu nvriy Grudziadzka University, Copernicus Nicolaus nformatics, ilc_5yr_v3.fits . ) ainmto ffidn pre- a finding of method lation . blt ftesae The soware. the of ability srslses oreproduce to easy results ts sfudta h fotin effort the that found as r sal aeavail- made usually are y oa eut facosmic a of results ional etemto described method the se r irr ucin for functions library ar, erdcinaecom- are reproduction e m cl.I hssense, this In scale. ime t ecie nSection in described lts eto fteUniverse, the of section n fRG8 Re- RBG08. of 3 and 2 s aamyb ifiutto difficult be may data dgtrpstr pack- repository git ed n20[3.Tepaper The 2008[23]. in tl ulse without published still resaelicences, free-soware neprr gfortran ontemporary rae ihaCfront C a with erfaced ri dniycd of code identity ArXiv hciei i n (ii). and (i) criteria th srpyiu eLo UMR5574, Lyon de Astrophysique ihn eeecsto references no with dc omctopol- cosmic a oduce ob,dsiebt (i) both despite be, to naédodecahedral incaré bropro,free- period, mbargo rainldt files data ervational t fti ae is paper this of ity e ym coau- my by ded . 5, 1 [¬Rp] Reproducibility of ’Poincaré dodecahedral space parameter estimates’

defining matched circles in the cosmic microwave background or matched discs in extragalactic 3-dimensional comoving space[21], is only recorded in the scientific lit- erature in papers published by my research group. To document and help analyse the success and difficulties in reproducing scien- tific results in this context, the editors of ReScience C posed the “Ten Years Repro- ducibility Challenge”, a request that scientists attempt to reproduce the main results of their own peer-reviewed scientific research papers that had been published before 1 January 2010, and document the method and results in ReScience C[18].

2 Method

The first steps for trying to reproduce the original results of RBG08 were to (re-)read the appropriate sections of the paper, initially taking the view of a non-author. 1. Section 2.1[23] states that the analysis method of Section 3.2 requires the three files at URLs listed in footnotes 1, 2, 3 on the same page. These files repre- sent two versions of an all-sky map of the Universe mostly representing cosmic −1 microwave background emission at 10h Gpc (comoving) from the Earth as observed by the Wilkinson Microwave Anisotropy Probe (WMAP)[11], and the “kp2” mask to enable analysis that avoids the most contaminated regions of the sky. These files need to be downloaded. 2. Footnote 7[23] indicates that circles-0.3.2.1, to be found at the URL http://cosmo. torun.pl/GPLdownload/dodec/, provides the soware for generating the figures and tables. This soware needs to downloaded from http://cosmo.torun.pl/GPLdownload/ dodec/circles-0.3.2.1.tar.gz. The next step was to develop a script on a git repository server that satisfies the re- quirements of the international scientific community, specifically the International Science Council[13], by not blocking access to scientists of any countries or territo- ries. During 2018 and 2019, several of the most popular git repository servers partially blocked access to scientists and other residents of several countries and territories (Github[10, 16, 26], Bitbucket[15], Gitlab[24]; the gitlab soware is free-licensed and can be installed independently of the Gitlab online service). The bans have presum- ably continued into 2020. A shi of my own soware to servers acceptable under in- ternational scientific ethical standards is underway, but incomplete as of early 2020. I chose a community-based server, Codeberg, not currently listed on the Wikipedia list of source code hosting facilities1. In 2019, the Investigating & Archiving the Scholarly Git Experience project team expressed its concerns about the bans, describing them as having “far-reaching and chilling consequences for open source, open scholarship, and for the open exchange of information and ideas”[12]. The remaining planned steps were to implement the minimal number of updates to make the code work and replicate the original results, using modern hardware and a modern soware environment. Footnote 7 of RBG08 warns that “These [circles- 0.3.2.1 and circles-0.3.8] and earlier versions of the soware require medium to ad- vanced GNU/Linux, Fortran77 and C experience for a scientific user.” There is no statement regarding the particular compiler(s) used. As far as I recall, it’s very likely that the widely used GNU fortran compiler of the time, g77, was used together with gcc, as selected automatically by autotool packages. The system and hardware chosen for the reproduction project were an AMD com- puter running with a Debian GNU/Linux 9.12 system on an x86_64 Linux-4.9.0 ker- nel. The Fortran compiler chosen was GNU Fortran (Debian 6.3.0-18+deb9u1) 6.3.0 20170516.

3 Results

The overall script intended to carry out the full sequence of downloads, configuring of packages, compiling of packages, subdirectory user-level installation of packages,

1https://en.wikipedia.org/wiki/Comparison_of_source-code-hosting_facilities

ReScience C 6.1 (#11) – Roukema 2020 2 [¬Rp] Reproducibility of ’Poincaré dodecahedral space parameter estimates’

ded213c1c4cfdfe2ef92f7155b27d58c wmap_ilc_5yr_v3.fits fbc8b2518fdddf0a1e7b5acde99a748e wiener5yr_map.fits 5aa3267dc6d69bf8c5f0a3a893e23960 wmap_kp2_r9_mask_3yr_v2.fits afbd67d8120c11e949eb0c414c2775f5 circles-0.3.2.1.tar.gz

Table 1. Checksums (md5sums) of the data and the main soware source code files of RBG08, use for the present reproducibility test.

setting up of calculation parameters, and running the main code, was set up as a bash script reproduce_RBG08.sh. The full package aiming to reproduce the figures and tables listed above is pro- vided at https://codeberg.org/boud/0807.4260, named aer the ArXiv identity of RBG08.

3.1 Downloading data and software source code 1. The URL in footnotes 1, 2 and 3[23] gave clickable links that were split into two and not correctly clickable. The user needs to cut/paste the two halves of each URL in order to obtain the three data files. The data files were downloaded with no apparent problem, with md5sums as indicated in Table 1. 2. The file circles-0.3.2.1.tar.gz with the md5sum indicated in Table 1 was down- loaded. It was included in the main git repository in its original form. Subse- quent changes are recorded in the git history at https://codeberg.org/boud/0807. 4260.

3.2 Compiling/debugging Fixes needed in order to successfully compile circles include: 1. A Fortran 77 line that ended on one line with a + symbol and started on the next line with another + symbol (within the valid columns for standard Fortran 77) was apparently accepted by the gcc family fortran compiler in 2008, but not now (2020). One of the + symbols was removed. 2. A fitting algorithm gsl_multifit_covar available in GNU Scientific Library (GSL) versions 1.x was obsoleted; it is no longer present in modern 2.x versions of GSL. With the aim of minimising the interventions required in the system, GSL- 1.10 was downloaded and compiled from source, re-creating part of the original soware environment. 3. Using the modern gfortran compiler options -fcheck=bounds -Wall to highlight likely sources of bugs due to insufficiently standard coding led to many warn- ings. Checking of these warnings motivated many fixes that could be expected to either solve errors in running the main circles package, or reduce the chance of calculational errors. 4. The autotools autoreconf command was run in the main circles-0.3.2.1 direc- tory and its subdirectories. 5. The cosmdist package provided by default in a subdirectory of circles-0.3.2.1 was replaced by a download/configure/compile/install section of the main re- production script, since cosmdist is now available in an online git repository. The aim was to reduce the chance of cosmdist being a blocking factor in repro- duction of the calculations. 6. Memory allocation errors that occur for running circles without previously definin- ing environment variables for key values such as input filenames are present in circles-0.3.2.1. These were most likely not noticed in RBG08 because of the use of environment variables providing these values. Fixes to the front-end C file circles.c were made with the intention of avoiding memory allocation er- rors, which are typically reported to the user as segmentation faults. However, memory allocation errors remained, most likely due to Fortran 77–C interfacing issues. This is described in the §3.4 on attempted running of the code.

ReScience C 6.1 (#11) – Roukema 2020 3 [¬Rp] Reproducibility of ’Poincaré dodecahedral space parameter estimates’

3.3 Dependencies The full list of dependencies listed – not necessarily really used – in the final gfortran compile operation that creates the circles binary executable is: -lpgplot -lpng lcosmdist -lisolat -lastromisc -llapack -lcblas -lf77blas -latlas -lgfortran -lquadmath -lcfitsio -lcosmdist -lgsl -lgslcblas -lm -lgcc -lX11 -lm.

3.4 Running and a software evolution block

Front end user-friendliness — At the time of writing the original paper, the aim was that the use of GNU tools to provide a free-licensed package configurable and compil- able with ./configure && make and a detailed ./circles --help command would be sufficient to enable easy reproduction by a scientifically competent user. For ex- ample, invoking the circles help option was intended to show both single-hyphen, one-character options and their equivalent double-hyphen, long options, such as -i, --cmb_file_raw=FILE cmb fits file of input data. The freshly compiled ver- sion of the code did this correctly, providing the user with a list of available options as expected.

Scripts — My private notes of what were intended to record the most significant steps taken in carrying out the project, along with more minor steps, were used in the at- tempted reproduction of RBG08. However, in trying to reproduce these steps now, it is clear that the original notes were not as complete and unambiguous as they should be. For the purpose of the current exercise in reproducibility, a completely fresh bash script was prepared, which verifies the sha512 checksums of input data files and so- ware packages that are downloaded from source rather than provided within the secu- rity context of the host system. The style of the new script is partly based on more re- cent attempts at reproducibility in my own recent papers in which git commit hashes of the soware[20] and a bash script for running the full soware[22] were provided. Some inspiration was taken from the make-based reproducibility framework[1] re- cently renamed maneage[2], but the structure of the script is much less modular; it is a simple linear script with a few minimal checks. This situation illustrates the problem of “insider knowledge” being required for the reproducibility of a paper, where “insider” also includes knowledge that may still be coded in the scientist’s brain, but not in written form.

Fortran 77–C interfacing — A more fundamental problem in terms of coding and so- ware environment evolution is that this code uses a C front end and a Fortran 77 backend, configured and compiled together using autoconf tools, in a way that, to the best of my knowledge and that of my co-authors, worked correctly in 2008. The key element for interfacing of Fortran 77 and C code that was recommended at the time was the use of AC_F77_WRAPPERS in the configure.ac file[8]. Three of the mod- ular packages called by the main code – cosmdist, astromisc and isolat – also use AC_F77_WRAPPERS. The main files are circles_f77.f, 2023 lines long, named to emphasise the ex- pected obsolescence of the Fortran 77 language standard; 21 Fortran 77 source files with a total of 11,616 lines of code in lib/; and 3415 lines of Fortran 77 code in the auxiliary package astromisc/lib/. For modernisation of this code, a minimum ap- proach would be to convert from Fortran 77 to Fortran 2008, in which case Fortran–C compatibility conventions that are reasonably well developed and implemented by the gcc/gfortran family could be used. Alternatively, f2c, which continues to be avail- able in distributions such as Debian GNU/Linux, could be used to convert the Fortran code into C. Any remaining bugs in the interfacing would be solvable within the stan- dard requirements of C, bypassing the issue of interlanguage communication. For the purposes of this reproducibility test, the required re-coding effort would be more than is presently justified. While this is, as far as I know, the only free- licensed code for identified-circles matching for the purposes of cosmic topology

ReScience C 6.1 (#11) – Roukema 2020 4 [¬Rp] Reproducibility of ’Poincaré dodecahedral space parameter estimates’

analysis for which peer-reviewed research has been published, the techniques de- veloped in codes that are not publicly available under either free or non-free licences have developed considerably since 2008. Moreover, the research field is extremely high risk: an observational confirmation of the spatial topology of the Universe would be a historically important discovery, but whether or not this measurement is feasible remains highly speculative. To serve as a basis for long-term projects in this partic- ular field, the soware would best be rewritten according to modern standards of C and/or Fortran; and the best tested, accurate, fast, well-coded free-licensed auxiliary libraries, such as cgal for geometrical purposes, could be used to avoid “reinventing the wheel”. Given that a hurried attempt at a major refactoring of the code would not only tend to extend beyond the scope of the aims of the “Ten Years Reproducibility Challenge”, it would also be pointless (a systematic upgrade would better be done properly and thoroughly), this reproducibility attempt was terminated, leaving it with an untraced Fortran–C interfacing memory bug.

4 Discussion

Compiler evolution and Fortran/C compatibility are the main elements of the diffi- culty in reproducing RBG08, together with the presence of some coding that was not sufficiently robust to allow for the interface change. In 2008, g77 was an obvious choice of Fortran compiler in the stable distribution of Debian GNU/Linux. Since the Debian community already had at the time a solid reputation in terms of soware security, verification of licensing and fully transparent and participatory decision- making, this seemed like a wise choice for reproducibility. Using a well-tested, widely used compiler and code standard seemed like a better long-term sustainable choice than using a compiler whose role was still being debated. The main developer of g77 had already announced his intention to stop maintaining the project in 2001[6], but even by 2010, two years aer RBG08 was published, the community remained unclear regarding the relationship between g95, based on gcc, versus gfortran, part of gcc[5]. However, also by 2010, gcc already claimed to implement compatibility between Fortran 2003 (ISO/IEC 1539-1:2004(E)) and ISO C99 (ISO/IEC 9899:1999)[9], imple- mented at the coding level by use ISO_C_BINDING and bind(C, ...) declarations and the ability to declare C types in a Fortran module. The IT group at the University of Oxford Department of Physics recommend the modern interfacing methods, and describe the g77 interfacing with C quite colour- fully, stating that “Part of the reason for the transition from g77 to gfortran is to make mixing-in with C code simpler, and avoid (most of) the acts of cruel and unusual pro- gramming which were previously required to get the compilers’ outputs to co-operate. Inevitably, the results of said acts were almost inevitably fragile and non-portable.”[4] An interesting question is whether a reproducibility framework such as maneage[2], which aims at a very high standard of reproducibility with maximum modularity and minimal dependencies, would have enabled easy reproducibility of the original project. The maneage system would, in principle, have configured, com- piled, and installed all the original soware environment, including GSL and g77 and a contemporaneous version of gcc, and would have encouraged the original project to be modular enough with sufficiently many verification tests to survive a decade of evolution of the soware environment. A key question would be whether a more modern version of gcc could have compiled the old versions of g77 and gcc. It is quite realistic to expect that a maneage reproduction of RBG08 would succeed more than the bash script method provided in this project. Would this project have been more easily reproducible had it been written in a higher level language, such as python? The question did not really arise at the time, since as is stated in the abstract of RBG08, one of the key results was a speed-up in computation time, a critical bottleneck for this type of research, in which case the computational overhead of higher level languages renders them impractical. The aim of the Ten Years’ Reproducibility Challenge is primarily to get “old code to run on modern hardware/soware (with minimal modifications)”. Nevertheless,

ReScience C 6.1 (#11) – Roukema 2020 5 [¬Rp] Reproducibility of ’Poincaré dodecahedral space parameter estimates’

an interesting question would be how much shiing back to an old soware environ- ment would be required to run the existing code. This raised a simple practical ques- tion: where can we find an official, archived version of the source code of g77 in the version likely to have been used when the paper was calculated and published? This turned out to be non-trivial, because the nature of the relation between g77 and gcc was not sufficiently known by me, nor was it easy to find using search engines. The Debian GNU/Linux developers’ guide to Fortran updating from g77 to gfortran, lasted edited in 2016, states that “g77 and g77-3.4 have been removed from the archive”[19]. It was only when the main developer of g77, James Craig Burley, kindly responded to a question posed using a social networking feature provided on a git repository server, that the packaging of g77 inside gcc tarballs became known to me. Thus, while finding g77 source code was not as easy as expected, once the detailed information about its relation with gcc was known, finding an archived version on a reputable webserver was found2. However, attempts at compiling g77-3.4.6 within gcc-3.4.6 on Debian GNU/Linux stable (9.12) were unsuccessful.

5 Conclusion

It is ironical that in the field of cosmic topology, not only are most soware pack- ages only available privately with unknown licences (as tends to be the case in as- tronomy as recently as 2015[3]), but the code that is explicitly free-licensed, publicly distributed and having peer-reviewed published results has turned out to be less easy to reproduce than expected. This is, unfortunately, consistent with typical reports on science research paper reproducibility[14, 7, 25]. The specific bottleneck suspected of leading to memory errors in this case was that the effort required to update the Fortran 77 files at the heart of the code, interfaced with a C front end, and com- piled with the current gfortran compiler from within gcc rather than with the older, discontinued[6] g77 compiler, risked being too great to be justified on any short time scale. While Fortran has remained actively used by scientists since more than half a century ago and in its modern standards continues to be used actively, and the original circles package was prepared using the powerful GNU autotools, a robust in- terface and standards for compiling C and Fortran code together have only evolved quite recently[9]. While the results of the paper are not as trivially reproducible as they appeared to be, the requirement of the Ten Years Challenge for the code to be placed in an online git repository, which in this case is https://codeberg.org/boud/0807.4260, resulted in con- firmation that the source code is fully free-licensed, including all libraries and other auxiliary soware packages, and the input data files remain publicly available online. Refactoring the code into a format such as maneage[1, 2] would be a potential way forward of shiing the research field towards a more completely open-science phase. Anyone interested in modernising the soware and completing the reproduction of the original results is welcome to contact the author of this paper.

Acknowledgments

Thank you to Konrad Hinsen (the reviewer) and James Craig Burley for several helpful com- ments. Part of this research has been supported by the “A next-generation worldwide quantum sensor network with optical atomic clocks” project of the TEAM IV programme of the Founda- tion for Polish Science co-financed by the European Union under the European Regional Devel- opment Fund. Part of this research has been supported by the Polish MNiSW grant DIR/WK/2018/12. Part of this research has been supported by the Poznań Supercomputing and Networking Center (PSNC) computational grant 314.

2https://ftp.gnu.org/gnu/gcc/gcc-3.4.6/

ReScience C 6.1 (#11) – Roukema 2020 6 [¬Rp] Reproducibility of ’Poincaré dodecahedral space parameter estimates’

References

[1] Akhlaghi M., Ichikawa T., 2015, The Astrophysical Journal Supplement Series, 220, 1 (arXiv:1505.01664)

[2] Akhlaghi M., Infante–Sainz R., Roukema B. F., Valls–Gabaud D., Baena–Gallé R., 2020, preprint (arXiv:2006.03018)

[3] Allen A., Teuben P. J., Ryan P. W., 2018, The Astrophysical Journal Supplement Series, 236, 10 (arXiv:1801.02094)

[4] Anonymous Oxford University Department of Physics 2020, Where has g77 gone to? ...or: libg2c considered harmful, Archived at Wayback, https://www2.physics.ox. ac.uk/it-services/where-has-g77-gone-to

[5] Bosscher S., Pfeifer G., Burnus T., 2010, The other GCC-based Fortran compiler, Archived at Wayback, https://gcc.gnu.org/wiki/TheOtherGCCBasedFortranCompiler

[6] Burley J. C., 2001, Why I’m Stopping My G77 Work, Archived at Wayback, http:// www.kilmnj.com/g77/why.html

[7] Chang A. C., Li P., 2015, Finance and Economics Discussion Series 2015-083,p.1

[8] GNU 2009, Fortran 77 Compiler Characteristics, Archived at Wayback, https://ftp. gnu.org/old-gnu/Manuals/autoconf-2.53/html_node/Fortran-77-Compiler.html

[9] GNU 2010, 7.1 Interoperability with C, Archived at Wayback, https://gcc.gnu.org/ onlinedocs/gfortran/Interoperability-with-C.html

[10] Github 2020, GitHub and Trade Controls, Archived at Wayback, https://help.github. com/en/github/site-policy/github-and-trade-controls

[11] Hinshaw G., et al., 2009, Astrophys.J.Supp., 180, 225 (arXiv:0803.0732)

[12] IASGE Team 2019, U.S. Sanctions Impact on the Git Community, Archived at Wayback, https://investigating-archiving-git.gitlab.io/updates/git-hosting- sanctions/

[13] International Science Council 2020, Freedoms & Responsibilities of Scientists, Archived at Wayback, https://council.science/what-we-do/freedoms-and-responsibilities- of-scientists/

[14] Ioannidis J. P. A., et al., 2009, Nature Genetics, 41, 149

[15] Jurdik S., Spaniol S., 2019, Iranian customers can’t open Confluence pages, Archived at Archive .today, https://community.atlassian.com/t5/Confluence-questions/ Iranian-customers-can-t-open-Confluence-pages/qaq-p/733404

[16] Liao R., Singh M., 2019, GitHub confirms it has blocked developers in Iran, Syria and Crimea, Archived at Archive.today, https://techcrunch.com/2019/07/29/github-ban- sanctioned-countries

[17] Luminet J., Weeks J. R., Riazuelo A., Lehoucq R., Uzan J., 2003, Nature, 425, 593 (arXiv:astro-ph/0310253)

[18] ReScience C Editors 2019, Ten Years Reproducibility Challenge, Archived at Way- back, https://rescience.github.io/ten-years

[19] Riku Nchip et al., 2016, GfortranTransition — Transitioning from g77 to gfortran, Archived at Wayback, https://wiki.debian.org/GfortranTransition

[20] Roukema B. F., 2018, Astronomy & Astrophysics, 610, A51 (arXiv:1706.06179)

[21] Roukema B. F., Kazimierczak T. A., 2011, Astronomy & Astrophysics, 533, A11 (arXiv:1106.0727)

ReScience C 6.1 (#11) – Roukema 2020 7 [¬Rp] Reproducibility of ’Poincaré dodecahedral space parameter estimates’

[22] Roukema B. F., Ostrowski J. J., 2019, JCAP, 2019, 049 (arXiv:1902.09064)

[23] Roukema B. F., Buliński Z., Gaudin N. E., 2008, Astronomy & Astrophysics, 492, 657 (arXiv:0807.4260)

[24] Smith D., 2018, Update on our planned move from Azure to Google Cloud Plat- form, Archived at Wayback, https://about.gitlab.com/blog/2018/07/19/gcp-move-update

[25] Stodden V., Seiler J., Ma Z., 2018, Proceedings of the National Academy of Sciences, 115, 2584

[26] Tung L., 2019, GitHub starts blocking developers in countries facing US trade sanctions, Archived at Wayback, https://www.zdnet.com/article/github-starts-blocking- developers-in-countries-facing-us-trade-sanctions

ReScience C 6.1 (#11) – Roukema 2020 8