Using Learning Data to Understand Student Engagement

• Consider data in context of the course, the situation and the student.
• Be aware of potential limitations of data: What it does and does not indicate.
• Learn how to record, view and setup attendance in Canvas.
• Learn about limitations with User-Level Analytics in Kaltura.
• Engage students as partners in improving learning environments:  Asking students questions about their behavior will help you interpret data correctly and clarify next steps
• Look for trends or patterns in performance associated with student work.
• Focus on instructional issues or questions
• Realize technologies are fallible and some data may be misleading.
• Remember FERPA considerations--data should be kept private and focused on furthering student success.

• Use a single data point for assessing student accountability.
• Rely on data as your sole teaching intervention.
• Share data in a way that reveals the identity of any individual student.
• Use tools (including Canvas and Kaltura) logs intended for diagnostics as a reliable record of user interaction. A myriad hardware, software, and connectivity issues can interrupt logging and analytics. 
• Use log data to prove a negative. If a log shows that a student used a tool, that is proof of engagement; however, the converse cannot be proved: if a student does not appear in the log, that does not prove that they did not engage with the tool.

Learning data can be accessed in Canvas within New Analytics using the following tabs. 

Course Grade allows you to view average grades for the course.

• Benefit: Provides a snapshot of how students in the course are performing overall and can help to pinpoint particular activities where students struggled or excelled.
• Caution: Ungraded and missing submissions are not included in the averages, which means these data may not reflect the performance of all students in the course.

Weekly Online Activity includes course “page views” and “participations.” 

• Benefit: Provides an indication of student interaction within the course. 
• Caution: Because page views only measure time logged into a page, not time-on-task, it may be most useful as an aggregate measure. Additionally, participations do not include communications or other activities that student(s) may have engaged in outside of Canvas.

Students allows you to view grades, activity, and communication data by student. 

• Benefit: Provides information about student performance, which can be used to take proactive steps to support student learning. 
• Caution: These data only offer a partial picture of student interaction within Canvas and not a comprehensive view of their efforts in the course.  

Reports generate CSV spreadsheets with data from New Analytics.

• The course access report is another resource available for individual student-level data. Similar to weekly online activity, these data represent time logged into a page, not time-on-task.

Learning data for videos you own or for which you are co-editor can be accessed in MediaSpace from the video’s media page under Analytics within the Actions menu. The data includes player impressions, number of plays, unique viewers, minutes viewed, and average completion rates.

• Benefit: Provides information about the number of people accessing a video and how long they engage with the video (drop off rate).

Caution: Individual user-level data is only available if students were logged into their UMN accounts and accessed the video through Canvas or a media channel within Kaltura MediaSpace.

Zoom offers reports for meetings you have hosted. These provide data on meeting attendance, duration, and poll results. 

• Benefit: Knowing the number of people joining your meetings and having a record of their participation (if you used polls during the meeting) can be useful to plan your synchronous class time.

Caution: Connectivity issues can cause a meeting participant to show up multiple times on the attendee list with their total duration in the meeting broken up over those entries. Joining breakout rooms also makes attendees appear to have left and rejoined the meeting.

Google Documents, Sheets and Slides all create a type of interaction history that provides data within the version history. Google Forms are often used to survey students and formally collect data. 

Version History

• Benefit: Knowing when a document was created, who worked on it, to what extent, and for how long can provide valuable information for assessing work.
• Caution: Participation history does not include communications or other activities the student(s) may have engaged in outside of the document or other documents or resources viewed.

Google Forms

• Benefit: Google Form responses are automatically summarized into helpful visual representations. Individual responses can be sorted.
• Caution: Data from responses is limited to questions asked. Robust survey design is complex and challenging.  

These types of Google data should not be confused with Google Analytics, a distinct tool offered by Google primarily used to track website traffic.