How MOOC Video Production Affects Student Engagement
March 12, 2014 | edX team
Videos are central to the student learning experience in many kinds of MOOCs.
The above figure shows four main kinds of videos on the edX platform:
a.) a recorded classroom lecture,
b.) an instructor’s talking head,
c.) a Khan-style digital tablet drawing (popularized by Khan Academy),
d.) a PowerPoint slideshow.
Question: How does video production affect student engagement in MOOCs?
How we went about it: We measured engagement by how long students watched each video and also whether they attempted to answer post-video assessment problems.
We took all 862 videos from four edX courses offered in Fall 2012 and hand-classified each one based on its type (e.g., traditional lecture, problem-solving tutorial) and production style (e.g., PowerPoint slides, Khan-style tablet drawing, talking head). We automatically extracted other features such as length and speaking rate (words per minute). We then mined the edX server logs to obtain over 6.9 million video watching sessions from almost 128,000 students.
To our knowledge, this is the largest-scale study of video engagement to date.
Here’s what we found and a couple recommendations:
Here are our seven main findings and corresponding recommendations for creators of online educational videos:
- Shorter videos are much more engaging. Engagement drops sharply after 6 minutes. Recommendation: Invest heavily in pre-production lesson planning to segment videos into chunks shorter than 6 minutes. This is the most significant recommendation!
- Videos that intersperse an instructor’s talking head with PowerPoint slides are more engaging than showing only slides. Recommendation: Invest in post-production editing to display the instructor’s head at opportune times in the video. But don’t go overboard because sudden transitions can be jarring. Picture-in-picture might also work well.
- Videos produced with a more personal feel could be more engaging than high-fidelity studio recordings.Recommendation: Try filming in an informal setting such as an office to emulate a one-on-one office hours experience. It might not be necessary to invest in big-budget studio productions.
- Khan-style tablet drawing tutorials are more engaging than PowerPoint slides or code screencasts.Recommendation: Introduce motion and continuous visual flow into tutorials, along with extemporaneous speaking so that students can follow along with the instructor’s thought process.
- Even high-quality prerecorded classroom lectures are not as engaging when chopped up into short segments for a MOOC. Recommendation: If instructors insist on recording traditional classroom lectures, they should still plan lectures with the MOOC format in mind and work closely with instructional designers who have experience in online education.
- Videos where instructors speak fairly fast and with high enthusiasm are more engaging. Recommendation:Coach instructors to bring out their enthusiasm and reassure them that they do not need to purposely slow down. Students can always pause the video if they want a break.
- Students engage differently with lecture and tutorial videos. Recommendation: For lectures, focus more on the first-time watching experience. For tutorials, add more support for rewatching and skimming, such as inserting subgoal labels in large fonts throughout the video.
Philip is an Assistant Professor of Computer Science at the University of Rochester. He received his Ph.D. in Computer Science from Stanford University in 2012 and was a visiting research scientist at edX from 2013 – 2014. His main research interests are in human-computer interaction (HCI), especially building tools for informal learning and online education. Learn more about his research in online education by visiting http://pgbovine.net/academic.htm or following @pgbovine on Twitter.