Final report to ACRL VAL Committee from Gap Analysis Subcommittee presented for ALA Annual Conference 2015

Recommendations for next steps posed in Connect, Collaborate and Communicate (2012 available at http://www.ala.org/acrl/sites/ala.org.acrl/files/content/issues/value/val_summit.pdf

Overall, the Gap Analysis Committee does not see a need at the present time to update the VAL Report, but we do recommend the following areas of new research.

LEARNING ANALYTICS

Q1.6 Develop strategies to advance library participation in learning analytics initiatives Q5.3 Investigate the potential incorporation and application of learning analytics practices in conjunction with ACRL Resources

We view these 2 recommendations as related and would like to treat them as a whole.

Although not widely discussed in the literature to date, we think learning analytics have great potential to assist libraries in using data to demonstrate library value. There are certainly privacy issues to consider in gathering and using this data, but with care and education on best practices for handling student data, the work can go forward.

We recommend that the ACRL VAL committee track this issue and incorporate it into the proposed webinar series on professional development programs in Objective 10 for the VAL committee this year. FYI – Megan Oakleaf is proposing a webinar this year on learning analytics basics.

To further this work, we recommend that a VAL subcommittee or task force be formed charged with continuing to follow the literature on learning analytics and to investigate possible eLearning opportunities and contributions to the Valueography as well as the VAL blog.

Please see attached bibliography on current issues in learning analytics.

We have talked with the ACRL Instruction Section and the ACRL Student Learning and Information Committee about collaborating on developing an approach to learning analytics. All three committees agreed to discuss this collaboration at ALA Annual 2015 and hopefully include it their annual work plans.

We also recommend tracking the work of Unizin (http://unizin.org)

From the website: Unizin services are cloud-based infrastructure based on open technology standards. It will evolve to support content systems that empower faculty with full control over their owned content – store it or share it- and an analytics service to enable research to improve learning/”

The founding members included Oregon State University, Colorado State University, the , the , the University of Wisconsin- Madison, the , , Pennsylvania State University, the and the .

Background:

At the beginning of our work, we did not have an intelligent and clear definition for learning analytics. We liked two definitions:

1) Steven Bell in “Keeping up with…learning analytics.” Available at http://www.ala.org/acrl/publications/keeping_up_with/learning_analytics

“Learning analytics refers to technologies, usually software tools, that enable the analysis of student data in order to identify learning weaknesses so that faculty, advisers and even librarians could intervene with corrective action.”

2) EDUCAUSE Center for Applied Research in “Analytics in Higher Education: Benefits, barriers, and recommendations.” Available at https://net.educause.edu/ir/library/pdf/ERS1207/ers1207.pdf

“Analytics is the use of data, statistical analysis, and explanatory and predictive methods to gain insights and act on complex issues.”

3) Additional definitions found in the literature

From: Elias. T. (2011). Learning analytics: Definitions, processes and potential. http://learninganalytics.net/LearningAnalyticsDefinitionsProcessesPotential.pdf

The identified goal of this emerging field is the “ability to scale the real-time use of learning analytics by students, instructors, and academic advisors to improve student success.” The focus appears to be on the selection, capture and processing of data that will be helpful for students and instructors at the course or individual level (p. 4)

From: Long, P. & Siemens, G. (2011). Penetrating the fog: Analytics in learning and education. EDUCAUSE Review, September/October

…Learning analytics already suffers from term sprawl… the ubiquity of the term analytics partly contributes to the breadth of meanings attached to it…Learning analytics is more specific than academic analytics: the focus of the former is exclusively on the learning process. Academic analytics reflects the role of data analysis at an institutional level, whereas learning analytics centers on the learning process (which includes analyzing the relationship between learner, content, institution and educator).

From Clow, D. (2012). An overview of learning analytics. Teaching in Higher Education, 18(6), 683-695

Most commonly cited definition of learning analytics was adopted by First International Conference on Learning Analytics and Knowledge in 2011,

Learning Analytics is the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs. (P, 685)

A key concern in learning analytics is the need to use the insights gathered from the data to make interventions to improve learning, to generate ‘actionable intelligence’ which informs appropriate interventions. (p. 685)

4). Related fields

Academic analytics

From: Von Barneveld, A. (2012). Analytics in higher education: Establishing a common language. ELI Paper 1.

A process for providing higher education institutions with the data necessary to support operational and financial decision-making.

From: Ferguson, R. (2012). The state of learning analytics in 2012: a review and future challenges. KMI Technical Report KMI-12-01. http://kmi.open.ac.uk/publications/techreport/kmi-12-01

We finally arrived at the term academic analytics as the encompassing term for our topic…. How academic enterprises use information to support decision-making. By using the term academic analytics we are not implying that we are only interested in academic decisions. … We are very interested in how institutions use data to make all sorts of financial and operational decisions. Nor are we studying how faculty uses data to perform research.

From Clow, D. (2012). An overview of learning analytics. Teaching in Higher Education, 18(6), 683-695

Two other emerging areas have significant overlapping with learning analytics. The first is academic analytics, which is the use of business intelligence in education. This tends to focus more at the institutional and national level, rather than on individual students and courses. (Chow, p. 685)

Educational Data Mining

From Clow, D. (2012). An overview of learning analytics. Teaching in Higher Education, 18(6), 683-695

The second is educational data mining (EDM), which seeks to develop methods for analyzing educational data and tends to focus more on the technical challenges than the pedagogical questions. (Chow, p. 685)

Business Intelligence

From Elias, T. (2011). Learning analytics: definitions, processes and potential. http://learninganalytics.net/LearningAnalyticsDefinitionsProcessesPotenti al.pdf

Business Intelligence is a well-established process in the business world whereby decision makers integrate strategic thinking with information technology to be able to synthesize “vast amounts of data into powerful, decision making capabilities.” (From Baker, B. 2007 dissertation, University of Md. Oct. 19, 2010.)

Action analytics

From: Noriss, D. (2008). Action analytics: Measuring and improving performance that matters in higher education. EDUCAUSE, 43(1),)

Deploying academic analytics to “produce actionable intelligence, service- oriented architectures, mashups of information/content and services, proven models of course/curriculum reinvention, and changes in faculty practice that improve performance and reduce costs.

5) We consulted with Kara on Q1.4 and this was her response:

Basically, these are predictive, early intervention systems at the individual level. Learning analytics tie to online data dashboards in real time showing individual student progress. So if a students hasn’t logged into the Blackboard in 10 days, or they show up on a faculty/advisor dashboard as yellow, we can assume they might be trouble because we know that students who don’t log in for 2 weeks are more likely to drop a class (I’m making this up, but you get the idea).

In the white paper, Karen and I reflected what we heard at the Summit where folks were just hearing of this and wondered if/how library data and student interactions could feed into those initiatives, i.e. database logins, circulation etc. It helps faculty and advisors know which students need help RIGHT NOW. (Email message from Kara Dec. 3)

6) Kara asked Karen Brown to help us with Q1.5:

This is really prompting librarians to ask: What data about student learning and success are being collected on my campus? Which campus departments/units have what type of data?

Would it be useful to the library in its assessment activities to have access to these sources of data? Are other campus departments/units aware of the data that the library collects that might contribute to their assessment activities? In other words, the library should be in communication with other departments/units about sources of data related to student learning and success.

Whether the campus defines the data as “big data” probably varies from campus to campus. Individual librarians should do this type of data source identification but any help they might get from an ACRL (or related) committee/task force about where the librarian may find different sources of data on his/her campus would probably be helpful.

As we wrote the report and reviewed discussions at the summits, it was clear the learner analytics is likely to become more prominent on campuses. The recommended action brings that topic to the table (i.e., academic library profession) for discussion. As Kara suggests, it could mean lots of things – maybe academic librarians reporting on the use of learner analytics on their campus; Or encouraging academic librarians to participate in higher education meetings, etc. related to learner analytics; Or working with ACRL to explore training / professional development options.

It may be that this type of data would be used in conjunction with ACRL Metrics (email message from Karen Brown, Dec. 9 )

Q1.6 Encourages librarians to be aware of the trend toward learner analytics in higher education in general, and the need for the profession and individual librarians to develop strategies for making sure that the library is part of learner analytics initiatives that may be occurring at their college/university (email message from Karen Brown Dec. 9)

Q5.3 Investigate the potential incorporation of and application of learning analytics practices in conjunction with ACRL Resources

6) Examples of libraries currently involved with learning analytics. We found relatively few examples of libraries currently using learning analytics in assessment,

In Megan Oakleaf’s EDUCAUSE/ELI webinar she discussed several institutions that are using learning analytics to identify students at risk on their campuses. We have not yet asked them to write a VAL blog on their work.

 University of Minnesota Libraries

– Integrated introductory library instruction in APLUS system. Advisors can make direct referrals to the library

http://www.ofyp.umn.edu/ofypmedia/fileshare/2015_FYC_Newsletter.p df

Also:

Soria, K. (2013). Library use and undergraduate student outcomes: New evidence for students’ retention and success. Portal: Libraries and the Academy, 13(2), 147-164.

Soria, K. (2014). Stacks, serials, search engines and students’ success: First- year undergraduate students’ library use, academic achievement and retention. Journal of Academic Librarianship, 40(1), 84-91.

 DePaul University Libraries

From Terry Taylor

I believe that Megan was referring to our integrating the library into the university’s new advising software, Bluestar. The vendor is actually Starfish but we rebranded it for DePaul. We’ve just started, but the initial step was to get library research consultations listed along with other support services that advisors could recommend to students as appropriate.

What we are looking at next is to begin having students swipe ID cards (as they do when they meet with writing and other tutoring) when they come to the library for a research consultation. Initially we will probably do the appointment-based consultations but the goal is to have students swipe when they come to the reference desk. We hope to do something similar to what U Minn did by connecting use of library research support to retention and GPA data.

 Nevada State College

– integrates library data (ezproxy data, study room use, instruction session attendance) to identify at-risk students

http://wcetconference.wiche.edu/session/how-it%E2%80%99s-madepredictive-analytics

http://bluecanarydata.com/

 Peirce College

7) Related Issues to consider - Privacy issues - Difficulties in getting access to data - Tools/dashboards for analysis - Tying learning analytics projects to Standards For Libraries in Higher Education

DATA SERVICES and OA

Q1.4 Identify common data sources on campuses Q1.5 Identify, describe and publicize data collection and management tools & systems.

1. We have contacted that current and incoming chair of the Research & Scholarly Environment committee to discuss possible collaboration between our committee on issues involving data services and OA 2. Please see attached bibliography on research data services that is part of the 2015 Environmental Scan. This document was provided to us by Minglu Wang of the ACRL Research Planning and Review Committee (RPRC) 3. We have also contacted the conveners of the ACRL Digital Curation Interest Group to talk about possible collaborative projects. They are interested in including us in their July, 2015 group conference call.