.png){kind=small}
The Following case study was produced in collaboration with Online Education Services (OES) and explores the benefit of using conversational online learning communities to foster student engagement and increase support for at-risk students. You can also view the case study on their website.
Summary:
Students observed in this study used Yellowdig Engage to build interpersonal connections that eased their anxieties around their studies, created a sense of community, and set them up for academic success.
Yellowdig participation was correlated to increases in student pass and retention rates, making it important to include in our machine learning models to identify at-risk students.
OES's bespoke Student Engagement Tool that relies on Yellowdig data highlights a student’s risk profile in real time, empowering academic and support staff to make direct contact to provide support.
Through the use of the Student Engagement Tool, an uplift in student success metrics was observed compared with a 9% increase in pass rates and a 7% increase in students progressing into their next study period, amongst the high-risk students.
Introduction
OES specializes in providing online program management and tailored education solutions, such as learning analytics, to improve experiences and outcomes for students. For over a decade, they have partnered with universities to solve their online, on-campus and blended learning challenges. They utilize leading-edge technologies to improve student performance, progression, and retention. One such technology is Yellowdig; a student engagement platform. It is implemented as an alternative to traditional discussion boards and the user-experience is similar to widely adopted social media platforms. Yellowdig allows instructors to create vibrant learning communities that are more interactive, engaging, and satisfying than traditional discussion boards.
This case study documents the relationship between Yellowdig participation and student outcomes. Student pass rates and retention per study area will be explored for three teaching periods in 2020, for one of OES's university partners.
Their university partner delivers online learning for students, most of whom are mature age and studying part-time.
Students at this partner institution use Yellowdig Engage as a safe place to talk about taking an online degree, often for the first time. They find support from the other students in the Community who are in similar life stages like being a parent or being back to college after a long time.
The case study concludes by discussing one of OES’s advanced analytics solutions to improve student performance and retention. We showcase a machine learning propensity model that identifies students at risk of dropping out, using critical engagement metrics and behavioral indicators, including Yellowdig Community participation. We also share the efficacy data of a tool that has been deployed in the Learning Management System (for academic use) and surfaced as a standalone web application (for central student support).
Why does OES promote Yellowdig communities to their students?
The Yellowdig Community is a student-to-student informal learning space used to provide an engaging platform for informal discussions between students from any course or discipline, creating a sense of belonging to the learning community. It is also used to promote initiatives and provide information to support students outside of the virtual classroom environment.
How are students participating in their university partner’s Yellowdig Community?
A ‘typical’ post in this Community is a student introducing themselves. The most used word in these posts is likely “nervous,” as the students consistently share about their anxiety of being an online university student with the pressures of working full time, having kids, or many other personal challenges many of their partner’s students face.
The students who are willing to share about their nerves are consistently met with numerous comments riddled with phrases like “I'm in a similar situation,” “I can empathize with how you are feeling,” and “I’m also very very nervous.”
The data shows there are 120 views on average per student-authored post, so these conversations are lending comfort not only to the individual students creating the posts, but also to the many students logging on to read the feed.
Some students also took on the role of “Influencers” and would post often with a topic called “Follow my Journey” where they shared about their lives as they were working to get their degree. They encouraged other students to use the “Follow” option, so they would be notified of their posts. They served as inspiration and encouragement for the other students on Yellowdig Engage.
The beauty of this Community is that it is student driven; it ebbs and flows with the needs of the students. First many students join as they prepare for a new term and introduce themselves and find study buddies, then throughout the semester they discuss specific needs like placements, tutors, grade worries, or even what to do if their learning management system is down, and then as the semester wraps up students ask about graduation and start the process of finding study buddies and friends again.
What does Yellowdig’s participation data reveal?
Yellowdig participation at OES's partner institution is optional and aims to facilitate informal discussion between students outside of the classroom.
The data shows that students are not only posting, but they are conversing, with a conversation ratio of 3.16. This ratio means that for every post there is on average around 3 comments.
Qualitative observation reveals that many students are building relationships that translate into real-life relationships: they find study buddies in their local area, they search for tutors in difficult subjects, and they seek out fellow students in their degree programs.
Figure 1 shows the participation rate by study tenure. It shows that participation is higher in the first teaching period of study, which highlights the critical need of the student cohort to find a sense of community right from the beginning of their degree. Participation in Yellowdig remains to be an important factor even as students get beyond 5 teaching periods because greater than 25% of students continue to engage in the Yellowdig platform without any grade incentive. For reference, the average social media app has an 8.7% 24-month usage retention rate, according to Statistica.
Figure 2 shows the participation rates broken down by course area, which highlights the inequity of participation across the study areas. At OES they use these granular metrics to identify study areas that will benefit from targeted initiatives to increase Yellowdig participation.
Can online communities uplift student performance?
OES's data indicates that there is a positive correlation between Yellowdig participation and student pass rates and retention. However, it is difficult to confirm causation without robust experimentation. It appears that a higher proportion of students that pass and are retained are engaged with Yellowdig Communities.
Figure 3 and Figure 4 show the improved retention and pass rates seen for the student cohort that use Yellowdig.
Data also indicates that there is a correlation between the level of Yellowdig participation and student results. Figure 5 highlights students who are viewing and commenting or posting in the community platform, in general, are achieving better results than students who don’t log in.
Predicting student performance and providing targeted support
As we discussed above there is correlation between Yellowdig participation and student pass rate and retention. With this knowledge, OES built predictive models to identify students at risk of dropping out or not performing well using Yellowdig data, integrated with other sources such as the LMS and CRM.
OES shared that the importance of Yellowdig participation to student performance makes it a vital feature of their machine learning classification models used to predict at-risk students. Below they have listed our top 5 features for one of their in-house machine learning risk models, where Yellowdig engagement is a key feature.
LMS Engagement Score
LMS Module Started Early
LMS Login Total Time
LMS Page Views
Yellowdig Activity
To enable access and usability of the predictive modeling by teaching and central support staff, they also developed a Student Engagement Tool that indicates a student’s risk profile in real time and provides functionality to make direct contact with the student (either through a phone call and/or SMS).
The tool is embedded into the LMS (for academic use) and surfaced as a standalone web application (for central student support) with close CRM integration, for the university partner discussed in this case study.
To ensure a successful implementation, teaching and support staff are empowered to take the right type of action, at the right time. Staff were provided training with the tool to decide which students would benefit from direct personal contact and what the best way to reach them would be.
Providing student support through the tool was initially piloted in selected undergraduate study areas across multiple teaching periods. To evaluate the success of the tool, students identified as high-risk in the intervention units were compared with a control group of high-risk students from previous iterations in the same study area and similar classes running within the same teaching period.
In each trial, approximately 25% of the target cohort were defined as high-risk and were candidates for intervention. Amongst the high-risk students, an uplift in student success metrics was observed compared to the control group in the majority of study areas. Overall, in the latest iteration, on average there was a 9% increase in pass rates and a 7% increase in students progressing into their next study period.
Final remarks
OES observed how Yellowdig Community engagement is positively correlated with favorable student outcomes and forms a key feature in machine learning models they develop. They found that Yellowdig Communities like their university partner’s foster genuine relationship building among adult learners and provide students with an outlet to express their anxieties to their peers. These Communities enable students to connect with students in their study area, and ultimately be more successful.
The predictive and advanced analytics they created to identify at-risk students requires data from the Learning Management System, Customer Relationship Management System, Student Management Systems, and third-party products such as Yellowdig. The combined data provides a holistic view of the student performance enabling the OES Student Engagement Tool’s machine learning modeling.
Their tool delivers the risk profiles of students to academic and student support staff members, for pre-emptive and personalized support, to improve student performance and retention.
Overall, Yellowdig’s Communities not only enable their students to feel more connected, but they have provided additional data insights that improve OES's ability to preemptively identify at-risk students.
