Data in Higher Education Series | Episode 14
Make it Count: Putting Your Data into Action
Published January 20, 2020
'Closing the loop' might be the most popular phrase in assessment—but are you actually making your assessment cycle count? In this episode we speak with Emily-Rose Barry, a senior product manager at Campus Labs, who lends her expertise to the idea of putting your data into action to make your assessment processes as seamless and effective as possible.
What are the common assessment needs across all accrediting bodies?
We’ve found that all accrediting bodies care about the use of data. It’s used for mission fulfillment first and foremost, but also resource constraints and reputation development. We need to care about public opinion—we’re in a political environment that challenges institutions to show their work and prove their worth and the use of data is crucial for that improvement. It’s the pretext and salvation for assessment.
Assessment is about taking the action, but everyone wants us to do it and it’s hard.
This isn’t new to anyone, but if we can use technology to help solve that problem then the use of data on campus is going to be so much easier for institutions to show.
What is the “act” phase?
We’ve found that nearly every institution we’ve talked to has some version of an assessment cycle. These cycles are methods of structuring assessment reporting to demonstrate continuous improvement. Campuses may have different names for the phases, but each of them can map back to a “Plan-Do-Check-Act” process as a longstanding framework. “Plan-Do-Check-Act” is the Shakespeare of assessment—it’s always been there and we just add new iterations. Ultimately, we are planning for assessment, doing work and collecting results, checking and interpreting those results and then coming back at the end of the cycle and acting on the impact of those changes.
Why is this cycle so important?
Closing the loop on use of data is arguably the most important thing our users are interested in and it’s also the most difficult. We decided to do some development related to this for a couple reasons.
- Institutional effectiveness professionals are spending way too much time checking in with people to see if they’ve done their assessment reporting.
- Shortening the feedback loop brings real potential to create a pathway between assessment finding and action.
We have data from 86 institutions with 2,300+ fields of information on their assessment cycles. 45% of those fields had to do with the “Plan” phase. Why do you think that is?
It’s the starting point and gets focus because we tend to think if we get off on the right foot, everything else will follow. It’s good for many people without a lot of assessment experience who might be responsible for assessment reporting on their campus. We also see folks struggling to make use of data, so maybe they didn’t ask the right question or use the right method and investing in that early stage is important to make sure they use the right tool to act on the problem.
But only 1 in 5 campuses in this sample set are actually talking about “actions taken.” How come?
There is a hierarchy of needs that can come into place with dealing with assessment processes. There is no black and white “this is wrong and this is right.” It’s more about understanding where your institution is right now and what your mission fulfillment is and that can be an evolving thing. There really isn’t an accreditor out there that says exactly what continuous improvement is and exactly how you can demonstrate it. Not everyone will define “Plan-Do-Check-Act” the same way as Campus Labs, but the way we have defined it poses questions people can ask themselves about their own processes to help evolve and improve them. There are some small changes that can be made to the structure for reporting that will allow people who have advanced to this level in their assessment processes—in both the program and department level—to let you know what they’re doing. That is the kind of work you should be highlighting for your accreditors.
What are some options for campuses to tweak their processes for the “Act” phase and improve the way they close the loop?
Campuses can add an additional step to their phase—Plan-Do-Check-Act-Follow up. Did you actually act and what was the outcome? The assessment committee members can also play a role in the follow-up, as well.
Another option is to adjust language and timing to be reflective when asking about actions taken. Instead of asking about proposed or recommended actions taken, only ask about actions that have been taken and what the impact of those actions are.
Finally, you can create an “actions-taken bank.” There are some nice resources out there that talk about meaningful actions like changes to curriculum or future assessment plan and changes to processes or staff training. When someone goes to document their actions taken, they can pull from the actions-taken bank which serves to inspire people for what’s expected and helps sort and categorize this information.