The Usage of Reference Management Software (RMS) in an academic environment : a survey at Tallinn University † Enrico Francese † Student at Digital Library and Learning Master Program – Oslo University College, Tallinn University, University of Parma

Abstract :This paper presents an online survey taken in research may 2011 at Tallinn University (TLU), Estonia, aiming to measure the usage of Reference Management 2. formatting bibliographies and citations when writing Software (RMS) in an academic environment. The papers sample embraces the whole corpus of TLU scholars: phd This last function is recently integrated in the latest students, researchers, teachers. A descriptive survey, version of the well-known text editors Microsoft Word or based on a constructivist approach, has been conducted OpenOffice.org/LibreOffice; this feature basically makes through an online questionnaire. RMS seem to suffer a less important the availability of a dedicated software to low spread among scholars; a general awareness is manage the coherence of references in a text. present, but more information and stronger support needs to be provided by libraries. The data collected can On the other hand, as RMS have become more be used as background for a deeper qualitative case sophisticated, their functionality has extended beyond study, and should also be compared to similar analysis the basic use for producing references in a consistent performed in different academic institutions through the style, and they may also offer tools for managing related rest of Europe. This survey is the first quantitative study documents (e.g. PDFs of the original paper you cite), or made on the subject. social networking tools, that allow, for example, to build, share and discuss collections of documents and citations Keywords: Reference Management Softwares, Citation among colleagues. Managers, Academic Libraries, Bibliography

DOI: II. LITERATURE REVIEW Although the nature and functions of RMS is quite Acknowledgements: This study has been conducted in considered by the disciplinary literature, there are few cooperation with the Library and Institute of Information previous researches about the effective usage of these Studies of Tallinn University. The author wishes to thank tools. prof. Sirje Virkus and ms. Sigrid Mandre from the Academic Library of Tallinn University for the help Martin (2009) and McMinn (2011) confirm two main provided and the constant assistance, without which this trends in the literature on RMS: training and technical paper would have not been possible. issues. The second one focuses mostly on the specific tools and technical aspects: features, tutorials and how- I. INTRODUCTION AND to's, quality, reliabaility, comparison and evaluation BACKGROUND (Dell'Orso, Fitzgibbons & Meert, 2010, Francese, 2011, In order to help authors manage large sets of references, “Innovations in Reference Management 1,” 2010, and to produce citations and references in a consistent “Innovations in Reference Management 2,” 2010). Steele style, a range of software packages is available. This (2008) focuses on the accuracy of the use of citations in type of software is often called ‘Reference Management research papers, providing usueful hints for those who Software', but ‘Citation managers’, ‘Bibliographic have to make decision about adoption of these softwares. management softwares’, 'Bibliographic softwares', or The second main trend consists in interesting CGC (Computer Generated Citation) are also used: informations about training initiatives that involved common examples are EndNote, RefWorks, Zotero. library staff (East, 2001, Siegler & Simboli, 2002). Olle I will use the term "Reference Management Software" & Borrego (2010) talk about scholars' behaviour, and the (from now on, RMS), defined as “a tool which enables importance of using a RMS to manage the increasing an author to build a library of references by entering the amount of electronic references. Martin (2009) writes details of each reference in a structured format. They about accuracy and pedagogical concerns, even though usually support mechanisms for organizing sets of he addresses tutors and instructors more than references by tagging, and will generate references, researchers. citations or bibliographies in a range of referencing Despite this wide interest for RMS distribution in styles" (Jisc, 2010). libraries, there are almost no studies about the To summarize, RMS have two main functions: connection between RMS and their usage. 1. building a database of citations, useful for keeping A lot of effort is put in training and promotion, but few track of and organize the documents useful for one's datas are provided circa the actual use and distribution. For example, Steele (2008) claims that "citation approaches research libraries take in providing similar management softwares exist since 1980 and are widely services: ensuring that the services provided are used today", but doesn't provide any reference for that. consistent with those of peer institutions; determining Kiernan (2006) provide only marketing statements from how services have been tailored to meet the unique the vendors, without a cross-check among clients. needs of different institutions; determining the level of support and optimum allocation of resources" (McMinn, The only survey about the usage of RMS seems to be the 2011). one made by Cibarelli in 1995 (Cibbarelli, 1995); her findings however must be considered out of date, since The research was conducted during the spring semester 16 years are a big leap in software development. A more of the International Master Program in Digital Library recent study by McGrath (2006) launches a new and Learning (DILL), taken place at the Tallinna Ülikool investigation, but his focus is stricly on students and (TLU) in Tallinn, Estonia. For this reason, I chose to solely related to RefWorks. address the specific closed community constituted by the TLU. This reason is motivated by the proximity of The role of libraries in providing support to scholars is access to the sample population and by the crucial help something which requires attention. According to East of a key-informant (my professor at TLU). (2001) "in many institutions the library has come to be seen as the main centre of expertise in matters related to Given the low amount of similar studies, the approach personal bibliographic softwares" (East, 2001). aims to be basic: I believe that a quantitative analysis is the first step to take to get a clear glance of the Kessler and Van Ullen (2005) also point at the role of phenomenon. For this reason a survey, made through an libraries, and reference librarians in particular, in online questionnaire, was the chosen method. This providing information and support on managing choice can be supported by Pickard, when she explains bibliographies and citations. Their focus on the less- that through a questionnaire "you can reach a large and expert scholars (undergraduate students) brings them to geographically dispersed community at relatively low state that a cautious approach must be used when cost, you can harvest data from a larger sample [...] and suggesting and promoting the use of RMS tools, since anonymity can be offered as well as confidentiality" they are not easy and require some expertise in citation (Pickard, 2007, p.183); another reason is the relative management. speed granted by this technique: due to the limited time of my permanence at TLU, a widebroad and extensive McMinn (2011) also stresses the importance of library data collection technique was necessary, although I was role. His survey about usage and distribution of RMS very aware of its intrinsic limitations: "the lack of takes as starting point the library support and training, so opportunity to talk directly to respondents" and the he questioned the libraries rather than addressing "notorious low response rate" (Pickard, 2007, p. 183). directly the users. His findings related to ARL libraries show a good commitment by library institutions in Finally, I wanted to measure also the impact of the promoting, licensing and providing training on RMS: library in the usage of RMS, so it was helpful to refer to "there are significant levels of support for bibliographic a community referring a single library (Tallinna Ülikooli management tools in major academic libraries as Akadeemiline Raamatukogu), rather than a larger determined by the number of libraries providing institution composed of several libraries. licensing, the level of instruction, and the creation of instructional materials and tutorials" (McMinn, 2011). IV. SAMPLE AND DATA COLLECTION

III. AIMS AND METHODOLOGY The questionnaire was sent to a list of 754 members of TLU, comprising PhD students, Researchers, Professors. The aim of this research is to gather practical Due to the nature of TLU, the sample mostly represents informations about the awareness and usage of RMS the areas of humanities and social sciences: it is to take among scholars. Moving from the literature exposed in account the fact that TLU doesn't have faculties of above, and from my personal experience and area of medicine, natural sciences and economics, so interests, I formulated the following research questions: considerations related to the disciplinary background can not be performed. 1) to which extent RMS tools are known and used by academics? The descriptive approach just aims not to provide deep analysis, rathen than to provide background for further 2) which softwares are mainly used? enquiry. Most of the questions were conceived as 3) is the library seen by scholars as a potential support in multiple dichotomous questions; only three open-ended the usage of RMS? questions were presented, with the purpose of adding some descriptive informations to integer the answers Collected objective data about the distribution and the obtained overall. Being mostly a quantitative study, the variety of approaches to the tool will be the first step to research doesn't go deep into the reasons for usage or understand the actual impact of RMS in scholars' non usage, or the different approaches. behaviour. McMinn explains quite convincingly the importance of this enquiry: "There are a number of A first pilot was administered to 8 scholars of TLU; the reasons why it is important to examine the different answers received allowed me to test the validity of the questionnaire and perform some minor tweaks. The tool users, almost the half is working with RMS since less adopted for the questionnaire was the Form function than 1 year. The number of citations collected, also in provided by Google Docs, which proved to be the best seldom high: 26% is below the number of 50. combination between cost (the tool is free) and easiness of use. The final questionnaire was then made available At this point is very important to state that the academic online from 30 april to 30 may 2011; the respondents library at TLU provides support to RefWorks. All TLU were reached by an email sent by the library to a list of students, faculty, staff and alumni have free access to TLU scholars who joined the ETIS network (Estonian RefWorks. Considered this, it is quite interesting to note Research Information System); to highten the attention how low are the responses concerning RefWorks: the of the recipients and encourage participation, the library 43% knows about its existence, yet only the 7% uses it. suggested to write the letter both in estonian and in Endnote by Thomson confirms to be the most widely english. A reminder was sent on the 17th of may. known instrument: the 70% of respondents knows about The data were collected anonimously, without any its existance, even though only 21% actually uses it. It's connection to the respondents. However an optional interesting to compare the usage of Endnote and Zotero. field was provided allowing respondents to leave their Though the former is more known than the latter, the contact if they wished to be part of further enquiries. The percentage of usage are the same: 21%. This can suggest high number of volounteers for this (29% of how a good free-of-cost alternative may often be respondents) witnesses the general interest of scholars in preferred to high-quality and high-expensive tools. helping with matters related to their bibliographic The social features of the RMS seem scarcely practice, and gives good hope for a further study which considered: only around 9% use RMS to share references would go on a deeper qualitative analysis of their among colleagues, or for discovery. The most common behaviour. usage is the managing of lists of references and their editing according to citation styles. The tools, when V. DATA ANALYSIS used, are still perceived on their basic functionality: the more modern social and web oriented approach seems to Population be not known, desired or pursued. The questionnaire got 58 responses, which constitute the When questioned about the reasons for non use, 7.7% of the sample. Low rate responses to online surveys are quite predictable (Pickard, 2007, p. 184); yet respondents claimed lack of knowledge and awareness (8 respondents), lack of skills and training (10), lack of this result can be interpreted as a first hint of a certain lack of interest or knowledge about the subject. It is interest, need and/or time (6). Some are basically anaware, but showed to be interested in knowing more, more likely that a person who doesn't know anything about RMS is not encouraged or inclined to take part in or to need just a little more support. An overall yet very fuzzy awareness is confirmed by the comments such a survey. provided: people don't seem to capture the potential of Due to the nature of the TLU, the answers came mostly the instruments, and a general confusion seems to be from humanities. The Estonian Institute of Humanities present ("I didn't understand clearly, how to use RMS"; provided the highest rate (19%), followed by the Insitute “It seems complicated for me”; “ they don't seem to of Education (11%); minor rates occurred in scientific integrate well with the way I work”). Finally, more than areas (mathematics and informatics 9%, psychology 7%, one respondents commented that they never heard about ecology 11%). The general response rate ranges from 0 the instrument. to 10%. Role of library Responses have a good distribution among academic roles: the most part are researchers (53%), but also A question about the role of libraries was made in order teachers and phd students are well represented. We must to give answer to one of the main research questions, as be aware that this question was open to multiple replies: inspired by the previous research literature discussed it is infact evident that roles can be overlapping in an above. academic research. The 11% says to get support from library; among those Age doesn't seem to be a factor in usage: the answers "I who didn't, the reasons vary: people can see RMS as don't use any RMS" are equally distributed among every quite easy to understand or consider the online learning age range. materials sufficient, others just complain about the lack of time. Nevertheless, a consistent part (25%) admit that Awareness and usage they didn't actually ask for support; so the relationship between library and users must be reconsidered. One The first evidence is given by the disproportion between respondent stated that no assistance was given in his the awareness and the usage. Respondents claim a case, but the library is always realiable. Judgement about general awareness of the existance of RMS: only 25% the libraries can be biased by the fact that the survey declare to know nothing about this type of tools. The came from the library staff. In any case, it seems that the effective non-usage, though, is high: 44% say that they library is seen as a realiable institute, but the support and don't use these instruments at all. Among the effective assistence are subjected more to the willingness of the individual scholar than to the ability or commitment of 150 (2010). the library staff itself. Francese, E., “Mendeley, dal social bookmarking al reference management: uno strumento innovativo VI. FURTHER RESEARCH per la gestione delle citazioni bibliografiche”, The study consititutes only a preliminary glance at the Biblioteche Oggi, 30(2) (2011). usage of RMS in academic environment. To obtain a Innovations in Reference Management 1., Innovations in merely numeric analysis was useful in a situation which Reference Management Event 1. Milton Keynes, seems to lack enough literature about it. These numeric (2010). analysis should be completed by a deeper enquiry which explores more qualitative aspects of the users' behaviour. Innovations in Reference Management 2., Innovations in Also it would be useful to widen the field of research to Reference Management Event 2. Birmingham, different disciplines, since this survey embraces only the (2010). scientific areas covered by TLU: the articles considered Jisc., Telstar (Technology Enhanced Learning supporting above, and the literature they review (see for example STudents to achieve Academic Rigour) Project. Lawrence & Ashwell, 1993) show that particular Retrieved November 28, 2010, from attention on RMS is given by the health-sciences http://www.jisc.ac.uk/whatwedo/programmes/institut community and libraries. ionalinnovation/telstar.aspx http://www.jisc.ac.uk/wh atwedo/programmes/institutionalinnovation/telstar.as CONCLUSIONS px (2010). Despite the tools adopted, the situation described by the Kiernan, V., “Toss Out the Indext Cards”, Chronicle of survey is one of shallow awareness. Higher Education, 52(40) (2006). The data collected show that RMS usage is low and not Lawrence, D., & Ashwell, S., “Reference management well supported by a proper knowledge. Scholars of software. Libraries can help you... and they do”, every role and age don't seem to be enough aware of the BMJ. British Medical Journal, 307(6903), 569 potential of these tools neither they have a good (1993). knowledge of their features and mechanism. The approach to it seems casual: despite the license and Martin, J. L., “Course instructor perceptions of computer- support provided on RefWorks by the library, only a generated bibliographic citations”, Reference small percentage of respondents uses this tool. This Services Review, 37(3), 304-312 (2009). means that the impact of library's communication is McGrath, A., “RefWorks investigated: An appropriate somehow limited, although the library is generally bibliographic management solution for health acknowledged of doing a good job in promoting and students at Kingʼ s College London?”, Library and providing assistence. To get the maximum efficiency by Information Research News, 30(94), 66-73 (2006). the library competencies, more official and continuous intiatives of literacy, information and training should be McMinn, H. S., “Library support of bibliographic taken on a regulare basis, and would constitute a good management tools: a review”, Reference Services investment by libraries. Review, 39(2), 278-302 (2011). Olle, C., & Borrego, A., “Librariansʼ Perceptions on the REFERENCES Use of Electronic Resources at Catalan Academic Cibbarelli, P. “Cibbarelliʼ s surveys: user ratings of Libraries: Results of a Focus Group”, New Library bibliographic citation management software”, World, 111(1), 46-54 (2010). Computers in Libraries, 15(4), 25-40 (1995). Pickard, A. J., Research methods in information. London: Dellʼ Orso, F., “Bibliography Management Software: Facet (2007). Analysis and Comparison of some packages”, Siegler, S., & Simboli, B., “EndNote at Lehigh”, Issues in Retrieved October 26, 2010, from Science and Technology Librarianship, 34 (2002). http://www.burioni.it/forum/dellorso/bms- dasp/text/index.html Steele, S. E., “Bibliographic Citation Management Software as a Tool for Building Knowledge”. East, J. W., “Academic Libraries and the Provision of Journal of Wound Ostomy & Continence Nursing, Support for Users of Personal Bibliographic 35(5), 463-468 (2008). Software”, LASIE: Library Automated Systems Information Exchange, 32(1), 64-70 (2001). Van Ullen, M. K., & Kessler, J., “Citation Generators: Generating Bibliographies for the Next Generation”, Fitzgibbons, M., & Meert, D. “Are Bibliographic Journal of academic librarianship, 31(4), 310-316 Management Software Search Interfaces Reliable?”, (2005). The Journal of Academic Librarianship, 36(2), 144-