Updated: May 26
The Yellowdig Team is currently preparing for our first annual Yellowdig Awards show. (Edit on 3/8/2021 - watch the recording here!) To determine award winners across different award categories (which include Best Community and Instructor of the Year), we created a composite Community Health metric that captures four key dimensions of student engagement and achievement. Along the way, we confirmed something we already knew: instructors who use many of our best practices have Communities that perform significantly better than instructors who use fewer.
New instructors using Yellowdig sometimes worry that our recommendations could lead to disaster in their courses, but our data indicate quite the opposite: it is riskier to ignore these practices.
However, we also discovered something new: following our best practices virtually guarantees success! New instructors using Yellowdig sometimes worry that our recommendations could lead to disaster in their courses, but our data indicate quite the opposite: it is riskier to ignore these practices.
Almost 100% of instructors who used all (or nearly all) of our best practices had online Communities that were well above the 50th percentile on Community Health.
Since we included every Yellowdig Community with at least 10 students and 20 combined Posts and Comments in this analysis (17,843 Communities at over 50 institutions), these outcomes can be expected regardless of institution type, student population, or other pedagogy decisions made by instructors. By contrast, most instructors who used few of our best practices scored below the 50th percentile.
Our other data (e.g., a report we did with ASU) show that Yellowdig activity and measures of Community Health like the conversation ratio (# of Comments per Post) are also related to grades and retention. Therefore, while it requires a slight shift in mindset from traditional discussions, aligning your pedagogy with our design and best practices is the least risky and most responsible way to use Yellowdig.
What is Community Health?
Our composite metric was derived from four dimensions of student engagement and achievement which typify successful Communities:
Conversation ratio, or the number of Comments divided by the number of Posts. The higher the conversation ratio, the more students are engaging in authentic, back-and-forth conversations with each other.
Earned proportion of 100% participation goal, or how many points students earned relative to the total participation goal. If this proportion is over 1.0, students earned more points than were required for a perfect Yellowdig grade. Meeting and exceeding the participation goal is a sign that students are more intrinsically motivated to participate. (In other words, they find value in the Community and are choosing to take part.)
Content per student, or the total number of Posts + Comments per student. The more students write, the more engaged they are, all else being equal.
Words over minimum, or the average number of words students wrote per Post/Comment in excess of the minimum word requirement for earning points. Like the earned proportion of 100% participation goal, this metric is a sign of intrinsic motivation to participate beyond requirements.
Each of these metrics was converted into a percentile score, and the mean of those percentiles became our Community Health metric.
Our Client Success team prides itself on having data-backed best practices that are open to revision in the face of new data. Creating a new Community Health metric for the Yellowdig Awards presented us with a new opportunity to put our best practices to the test. We plotted the composite Community Health score for every Community in Yellowdig with at least 10 students and 20 combined Posts and Comments.
Is there a relation between a Community’s Health Score and the number of best practices instantiated by the instructor?
The answer is a resounding yes. Even more interesting (and perhaps surprising to those who find some of our suggestions unorthodox) is that instantiating more best practices also increased baseline Community Health. In other words, using our best practices not only increases your likelihood of success (seen in the upward slope of the green line in the graphs below), but also practically eliminates the possibility that your Community will fail (seen through the complete absence of blue dots below the red line in the lower-right quadrant of both graphs).
Using our best practices not only increases your likelihood of success, but also practically eliminates the possibility that your Community will fail.
This effect was even more pronounced in large Communities (def. 100+ board followers) than in small Communities (def. 10 to 100 board followers). We think this is because professors have a bigger “voice” in small Communities and can therefore impact it more, both positively and negatively. Instructors in small Communities can also employ time-consuming strategies for managing individual student behavior that are unnecessary and become untenable in larger classes. Our data would suggest that these strategies likely backfire as often as they succeed. Regardless, rather than being a gamble, embracing our best practices is by far the safest course of action and will all but guarantee above-average outcomes.