VAL Cmte Gap Analysis Final Recommendations
Total Page:16
File Type:pdf, Size:1020Kb
Final report to ACRL VAL Committee from Gap Analysis Subcommittee presented for ALA Annual Conference 2015 Recommendations for next steps posed in Connect, Collaborate and Communicate (2012 available at http://www.ala.org/acrl/sites/ala.org.acrl/files/content/issues/value/val_summit.pdf Overall, the Gap Analysis Committee does not see a need at the present time to update the VAL Report, but we do recommend the following areas of new research. LEARNING ANALYTICS Q1.6 Develop strategies to advance library participation in learning analytics initiatives Q5.3 Investigate the potential incorporation and application of learning analytics practices in conjunction with ACRL Resources We view these 2 recommendations as related and would like to treat them as a whole. Although not widely discussed in the literature to date, we think learning analytics have great potential to assist libraries in using data to demonstrate library value. There are certainly privacy issues to consider in gathering and using this data, but with care and education on best practices for handling student data, the work can go forward. We recommend that the ACRL VAL committee track this issue and incorporate it into the proposed webinar series on professional development programs in Objective 10 for the VAL committee this year. FYI – Megan Oakleaf is proposing a webinar this year on learning analytics basics. To further this work, we recommend that a VAL subcommittee or task force be formed charged with continuing to follow the literature on learning analytics and to investigate possible eLearning opportunities and contributions to the Valueography as well as the VAL blog. Please see attached bibliography on current issues in learning analytics. We have talked with the ACRL Instruction Section and the ACRL Student Learning and Information Committee about collaborating on developing an approach to learning analytics. All three committees agreed to discuss this collaboration at ALA Annual 2015 and hopefully include it their annual work plans. We also recommend tracking the work of Unizin (http://unizin.org) From the website: Unizin services are cloud-based infrastructure based on open technology standards. It will evolve to support content systems that empower faculty with full control over their owned content – store it or share it- and an analytics service to enable research to improve learning/” The founding members included Oregon State University, Colorado State University, the University of Minnesota, the University of Michigan, the University of Wisconsin- Madison, the University of Iowa, Indiana University, Pennsylvania State University, the Ohio State University and the University of Florida. Background: At the beginning of our work, we did not have an intelligent and clear definition for learning analytics. We liked two definitions: 1) Steven Bell in “Keeping up with…learning analytics.” Available at http://www.ala.org/acrl/publications/keeping_up_with/learning_analytics “Learning analytics refers to technologies, usually software tools, that enable the analysis of student data in order to identify learning weaknesses so that faculty, advisers and even librarians could intervene with corrective action.” 2) EDUCAUSE Center for Applied Research in “Analytics in Higher Education: Benefits, barriers, and recommendations.” Available at https://net.educause.edu/ir/library/pdf/ERS1207/ers1207.pdf “Analytics is the use of data, statistical analysis, and explanatory and predictive methods to gain insights and act on complex issues.” 3) Additional definitions found in the literature From: Elias. T. (2011). Learning analytics: Definitions, processes and potential. http://learninganalytics.net/LearningAnalyticsDefinitionsProcessesPotential.pdf The identified goal of this emerging field is the “ability to scale the real-time use of learning analytics by students, instructors, and academic advisors to improve student success.” The focus appears to be on the selection, capture and processing of data that will be helpful for students and instructors at the course or individual level (p. 4) From: Long, P. & Siemens, G. (2011). Penetrating the fog: Analytics in learning and education. EDUCAUSE Review, September/October …Learning analytics already suffers from term sprawl… the ubiquity of the term analytics partly contributes to the breadth of meanings attached to it…Learning analytics is more specific than academic analytics: the focus of the former is exclusively on the learning process. Academic analytics reflects the role of data analysis at an institutional level, whereas learning analytics centers on the learning process (which includes analyzing the relationship between learner, content, institution and educator). From Clow, D. (2012). An overview of learning analytics. Teaching in Higher Education, 18(6), 683-695 Most commonly cited definition of learning analytics was adopted by First International Conference on Learning Analytics and Knowledge in 2011, Learning Analytics is the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs. (P, 685) A key concern in learning analytics is the need to use the insights gathered from the data to make interventions to improve learning, to generate ‘actionable intelligence’ which informs appropriate interventions. (p. 685) 4). Related fields Academic analytics From: Von Barneveld, A. (2012). Analytics in higher education: Establishing a common language. ELI Paper 1. A process for providing higher education institutions with the data necessary to support operational and financial decision-making. From: Ferguson, R. (2012). The state of learning analytics in 2012: a review and future challenges. KMI Technical Report KMI-12-01. http://kmi.open.ac.uk/publications/techreport/kmi-12-01 We finally arrived at the term academic analytics as the encompassing term for our topic…. How academic enterprises use information to support decision-making. By using the term academic analytics we are not implying that we are only interested in academic decisions. … We are very interested in how institutions use data to make all sorts of financial and operational decisions. Nor are we studying how faculty uses data to perform research. From Clow, D. (2012). An overview of learning analytics. Teaching in Higher Education, 18(6), 683-695 Two other emerging areas have significant overlapping with learning analytics. The first is academic analytics, which is the use of business intelligence in education. This tends to focus more at the institutional and national level, rather than on individual students and courses. (Chow, p. 685) Educational Data Mining From Clow, D. (2012). An overview of learning analytics. Teaching in Higher Education, 18(6), 683-695 The second is educational data mining (EDM), which seeks to develop methods for analyzing educational data and tends to focus more on the technical challenges than the pedagogical questions. (Chow, p. 685) Business Intelligence From Elias, T. (2011). Learning analytics: definitions, processes and potential. http://learninganalytics.net/LearningAnalyticsDefinitionsProcessesPotenti al.pdf Business Intelligence is a well-established process in the business world whereby decision makers integrate strategic thinking with information technology to be able to synthesize “vast amounts of data into powerful, decision making capabilities.” (From Baker, B. 2007 dissertation, University of Md. Oct. 19, 2010.) Action analytics From: Noriss, D. (2008). Action analytics: Measuring and improving performance that matters in higher education. EDUCAUSE, 43(1),) Deploying academic analytics to “produce actionable intelligence, service- oriented architectures, mashups of information/content and services, proven models of course/curriculum reinvention, and changes in faculty practice that improve performance and reduce costs. 5) We consulted with Kara on Q1.4 and this was her response: Basically, these are predictive, early intervention systems at the individual level. Learning analytics tie to online data dashboards in real time showing individual student progress. So if a students hasn’t logged into the Blackboard in 10 days, or they show up on a faculty/advisor dashboard as yellow, we can assume they might be trouble because we know that students who don’t log in for 2 weeks are more likely to drop a class (I’m making this up, but you get the idea). In the white paper, Karen and I reflected what we heard at the Summit where folks were just hearing of this and wondered if/how library data and student interactions could feed into those initiatives, i.e. database logins, circulation etc. It helps faculty and advisors know which students need help RIGHT NOW. (Email message from Kara Dec. 3) 6) Kara asked Karen Brown to help us with Q1.5: This is really prompting librarians to ask: What data about student learning and success are being collected on my campus? Which campus departments/units have what type of data? Would it be useful to the library in its assessment activities to have access to these sources of data? Are other campus departments/units aware of the data that the library collects that might contribute to their assessment activities? In other words, the library should be in communication with other departments/units about sources of data related to student learning and success. Whether the campus defines the data as “big data” probably varies from campus to campus. Individual