Data Analysis: Chat Service Usage During Remote Instruction
I analyzed usage patterns of a library chat service to identify what changes occurred after the switch to remote learning in early 2020. I published results on a blog for other Libraries staff.
Purpose:
I created this project to see how library chat services were used different after UNCC-Chapel Hill made the switch to online classes in March 2020. Reference chat transcripts are stored on a service called LibraryH3lp, which also allows those with access to tag chats manually and pull statistics on usage patterns. The project became a blog post visible to staff at UNC-Chapel Hill Libraries.
The end goal was to present to the UX department and to other stakeholders through an online post as part of an analysis of the usefulness of various features in the UNC-Chapel Hill Libraries’ website. This was a part of the usability testing performed before launching a full website redesign.
Process:
I compared chat statistics from April 2019 to April 2020 (the first full month of remote instruction at UNC) to see what differences there were in usage statistics and types of questions asked. First, I compared the monthly usage trends from 2019 to 2020. Then, in terms of more specific usage statistics, I addressed how peak usage hours shifted, how chat durations changed, how channel usage shifted, and how users found the chats. I performed statistical analyses on this data in Microsoft Excel.
Outcomes:
The result of this project was a blog post to share my findings with others in my UX department and stakeholders from departments, such as those in charge of staffing decisions for these chat services. This was a part of the usability testing performed before launching a full website redesign.
Overall, the main changes that the chat services have seen in the month after the transition to remote learning include:
Less evening usage
Higher traffic overall
More questions about finding particular resources
More questions asking for help with electronic resources or subscription services
Lessons Learned:
This project was an exciting dive into informal research: research that was informative but not conducted with official, peer-reviewed publication in mind, so I could examine more variables all at once. It was helpful to write this for a non-UX audience because it helped me to articulate the purpose of each of the statistical measures I was performing.
If I were to do this project again, I would want to find a better way to tag chat transcripts than doing so by hand. I considered using a software that I had access too, but I would still have to read each of the transcripts myself.