How Responsiveness to Data Created Evaluation Success
When the University of Houston-Downtown, Marilyn Davies College of Business (MDCOB) partnered with Campus Labs in 2013, they adopted Course Evaluation with IDEA as their tool for measuring teaching effectiveness across all college programs within the institution. In support of their newly adopted tool, they developed a consistent and dedicated approach to their course evaluations process by illustrating the use of data-informed decisions at the campus level.
In 2016, with three years of valuable feedback to reflect upon, MDCOB created two offices to help guide the college in using data to inform course and program improvement and faculty development.
The Office of Instructional Excellence (OIE) was established to enhance teaching effectiveness of MDCOB faculty. Led by an Executive Director of Instruction and Faculty Development who holds a PhD in Curriculum and Instruction and has 20 years of experience in higher education, including tenure-track faculty positions in teacher education, the OIE reviews student opinion surveys, programmatic assessment results, and course assessment results as a basis for continuous improvement of instruction in the College. Some of the OIE’s responsibilities include:
- Developing learning objectives and creating course elements/assessments that align with the stated objectives
- Developing rubrics to assess student learning
- Designing new online courses and/or translating existing on-campus courses for the online environment
- Developing mid-term course evaluations (student surveys)
- Selecting objectives for student ratings of instruction (IDEA) and identifying teaching strategies aligned with those objectives
- Interpreting data from student ratings of instruction (IDEA) to identify instructional/curricular strengths and needs and develop strategies for making the needed changes
The Office of Business Analytics & Decision Support (OBADS) was established to enhance data-driven decision support in the MDCOB. Led by a Director of Business Analytics who holds a Master of Arts in Computational and Applied Mathematics and nearly 20 years of experience in higher education, studying student enrollment, persistence, and completion trends, the OBADS provides support to OIE in teaching and learning related initiatives and projects that draw on data. Some of the responsibilities of OBADS include:
- Equipped with a Qualitative Researcher and staff trained in data analytics
- Collaborates with colleagues in the MDCOB to enhance data-driven decision making
- Develops a set of dashboards to individuals to provide a visual on activities against established goals
- Works with colleagues to provide a consistent set of measures to evaluate continuous improvement
- Provides support for various accreditation submissions
Staff from the OIE and OBADS work collaboratively to facilitate meaningful use of Campus Labs IDEA data for faculty development and instructional effectiveness.
The proof is in the numbers
- The systematic approach has resulted in course/program improvement & faculty development that is built on efficiency and effectiveness
- Faculty showing more buy-in
- Response rates have risen from 15% to over 40% at the undergraduate level
- Response rates have risen from 40% to over 60% at the graduate level
Data Leads to Action
For the 2016-2017 academic year, the course evaluation data revealed that not all the MBA classes made effective use of team-teaching that is a hallmark of their graduate education. These courses involve a faculty member and an industry expert “Corporate Fellow” that creates a dynamic experience of discussion, debate, sharing of content expertise, etc. With this information in hand, the OIE embarked on a qualitative study, using classroom observations and interviews as data sources, and was able to identify elements of effective faculty and corporate fellow team-teaching partnerships. The resulting guidelines now serve as reference material for faculty and corporate fellows.
In 2018, the IDEA/Campus Labs Course Evaluation tool revealed student dissatisfaction with online courses (there was a lack of interaction). This prompted the OIE to hold webinars to inform faculty of the available tools and pedagogical practices they can access to spark engagement and interaction with their online classes.
Course Evaluation data also revealed that the use of digitized lectures produced by faculty was sub-par. They were often too basic and did not have the power to make learning content memorable. This data moved the MDCOB to make investments to improve the video production room capabilities through the following improvements: the purchasing of a green screen and teleprompter; a graphic designer was hired to translate lecture content into graphic elements and animations; and guidelines for faculty were also crafted.
Empowering Faculty and Students Through Data
Since the University of Houston-Downtown MDCOB is a centrally located campus, it serves a strong majority of first-generation students. Their keen focus on student feedback has allowed for data-informed decisions to be made on campus while keeping the student at the heart of the mission. According to Sedef Smith, PhD, Executive Director of Instruction and Faculty Development, “After seeing firsthand how a scientific method towards IDEA-Campus Labs survey results (through qualitative and quantitative analysis) informs teaching and instruction, MDCOB faculty are showing more buy-in, as evidenced by student response rates. The particular characteristics of the tool, such as its online accessibility and prompt generation of results, have supported the achievement of [our] goals.”
Katie received her PhD in American Studies from the University at Buffalo in 2016. She has worked as an adjunct in the disciplines of history, American studies and American popular music for more than 10 years, teaching a wide variety of undergraduate courses for Buffalo State College, Hilbert College, Southern New Hampshire University, SUNY Fredonia, and University at Buffalo. Additionally, Katie holds a master’s degree in history from Buffalo State College and bachelor’s degree in history from SUNY Fredonia. As a consultant, Katie leverages her teaching and research experiences by working primarily with Course Evaluations campuses. When she isn’t talking about best practices for teaching and learning, she is likely sharing funny antics of her two young children or drinking coffee!