Updated: Nov 4, 2021
The Yellowdig Team is currently preparing for our first annual Yellowdig Awards show. (Edit on 3/8/2021 - watch the recording here!) To determine award winners across different award categories (which include Best Community and Instructor of the Year), we created a composite Community Health metric that captures four key dimensions of student engagement and achievement. Along the way, we confirmed something we already knew: instructors who use many of our best practices have Communities that perform significantly better than instructors who use fewer.
New instructors using Yellowdig sometimes worry that our recommendations could lead to disaster in their courses, but our data indicate quite the opposite: it is riskier to ignore these practices.
However, we also discovered something new: following our best practices virtually guarantees success! New instructors using Yellowdig sometimes worry that our recommendations could lead to disaster in their courses, but our data indicate quite the opposite: it is riskier to ignore these practices.
Almost 100% of instructors who used all (or nearly all) of our best practices had online Communities that were well above the 50th percentile on Community Health.
Since we included every Yellowdig Community with at least 10 students and 20 combined Posts and Comments in this analysis (17,843 Communities at over 50 institutions), these outcomes can be expected regardless of institution type, student population, or other pedagogy decisions made by instructors. By contrast, most instructors who used few of our best practices scored below the 50th percentile.
Our other data (e.g., a report we did with ASU) show that Yellowdig activity and measures of Community Health like the conversation ratio (# of Comments per Post) are also related to grades and retention. Therefore, while it requires a slight shift in mindset from traditional discussions, aligning your pedagogy with our design and best practices is the least risky and most responsible way to use Yellowdig.
What is Community Health?
Our composite metric was derived from four dimensions of student engagement and achievement which typify successful Communities:
Conversation ratio, or the number of Comments divided by the number of Posts. The higher the conversation ratio, the more students are engaging in authentic, back-and-forth conversations with each other.
Earned proportion of 100% participation goal, or how many points students earned relative to the total participation goal. If this proportion is over 1.0, students earned more points than were required for a perfect Yellowdig grade. Meeting and exceeding the participation goal is a sign that students are more intrinsically motivated to participate. (In other words, they find value in the Community and are choosing to take part.)
Content per student, or the total number of Posts + Comments per student. The more students write, the more engaged they are, all else being equal.
Words over minimum, or the average number of words students wrote per Post/Comment in excess of the minimum word requirement for earning points. Like the earned proportion of 100% participation goal, this metric is a sign of intrinsic motivation to participate beyond requirements.
Each of these metrics was converted into a percentile score, and the mean of those percentiles became our Community Health metric.
Our Client Success team prides itself on having data-backed best practices that are open to revision in the face of new data. Creating a new Community Health metric for the Yellowdig Awards presented us with a new opportunity to put our best practices to the test. We plotted the composite Community Health score for every Community in Yellowdig with at least 10 students and 20 combined Posts and Comments.