Photo credit: https://www.eposts.co/what-are-the-types-of-culture/
So I am sure you have heard the talk? I am referring to the talk about culture and assessment. Culturally responsive assessment has emerged in conversations among the student affairs Assessment Leaders, NASPA Multicultural Institute and the National Conference on Race and Ethnicity in American Higher Education (NCORE) groups. These conversations have centered on the importance of ‘inclusion’ with regards to facilitating and doing effective assessment. The National Institute for Learning Outcomes Assessment (NILOA) recently published an article, Equity and assessment: Moving towards culturally responsive assessment by Montenegro & Jankowski (2017), in which the authors outline a set of best practices and recommendations for upholding culturally-responsive assessment practice within the classroom to add to the conversation.
Fortunately, in student affairs, best practices for assessment already use the recommendations outlined in the article such as ‘putting students first’ and ‘meeting students where they are’. However, in order to build an authentic culture of assessment that is sustainable, it is important to go beyond by expanding our scope of what we mean by assessment outside a typical course-learning environment.
The article discusses the critical ability to look at specific student subpopulations and truly understand where these students are coming from, with the goal to build assessment that best allows students to demonstrate their unique learning. The goal, they argue, is to ensure that our classroom assessment practices are flexible to incorporate multiple methods for students to demonstrate what they learn in the context of the course. However, it does not discuss how this plays out across the entire student experience.
The article expresses some great practices in regards to course-level assessment and many of these same practices are in important within the framework of student affairs assessment. But how exactly do we build this authentic culture? Student affairs assessment offices are uniquely positioned to become experts in the many topics of our diverse institutions. Because of our vast assessment and research practices from advisement and crisis intervention to leadership and engagement, we have a very holistic view of the student experience. We can cultivate culturally-responsive assessment by leveraging and best integrating our knowledge regarding multiple facets of the student experience within our assessment approaches. We can be strong liaisons on our campus to influence, lead and encourage culturally responsive assessment practices.
So how do we do this in practice...
Emily Morter via Unsplash
Tip #1: Focus on the Who
Leaders often want to immediately understand if we are doing things well. Instead, take a few steps back and focus on the ‘who we are serving’ and if we are doing the right things. The right things will vary based on institutional context, but we should be focusing on the core purpose of the experience and begin assessment from there rather than jumping directly to the results. Sometimes it is a matter of challenging the why and who. Sometimes leaders want to understand if their organization is doing things well. Who is setting those priorities? Sometimes it is a matter of reflecting on the social justice orientation of the leaders who are setting those priorities. However, let’s continue to be inclusive in our focus on who we are serving by involving many key constituencies on our campuses in these conversations. For example, if a program is dedicated to peer-mentoring for freshmen with the goal to acclimate them to a college campus, this is the core purpose and the why. To go a step further and ensure this program is meeting the needs of students from various backgrounds, it is critical to focus on the who when designing the assessment and understanding impact. Focusing on demonstrating the value-added of peer-mentoring for these students and if that varies within and between student subgroups is just as critical. This is an important part of the culturally responsive assessment process with the goal to build, expand, and improve this program for all, as well as continue to meet the unique needs of different student populations.
Fleur Treuniet via Unsplash
Tip #2: Measurement is Critical
In order to build a culturally responsive assessment culture, there is a strong need to provide reliable and valid evidence regarding practices. There are discussions in our work regarding the narrative of reliability and validity. I would argue that it is important to be more holistic in what we define as reliable and valid. Measurement is one area that needs priority in our work. What is key here is that culturally responsive assessment doesn’t abandon measurement, but in fact it embraces it along with other qualitative efforts to truly understand the student experiences from all groups.
The article mentions research indicating that you cannot use the same measure to demonstrate learning outcomes, because it is not “equally valid.” However, if using structural equation modeling (SEM) techniques, such that of using a measurement invariance, these analytics are structured to understand how the same construct is measured with different subgroups of respondents. The goal of this technique is to determine if there is measurement invariance and understand if the measures are equally valid. This is a key step before deciding not to use a specific measure. It is critical to not just omit a measure without going through the steps.
In fact, multiple measures are important so we can understand many different facets of the student experience. One can utilize many different measurement tools to understand more holistically about the student experience. So before you decide to not use a measure, go through the measurement steps, make this a priority and use multiple measures to understand student learning, development, and the value-added of their experience.
Ian Schneider via Unsplash
Tip #3: Work with Community to Bridge the Gap
Implementing a culturally responsive assessment culture extends beyond process and measurement, but focuses on building community around these efforts. There is value in having diverse faculty, staff, and students engaged in the process not only to challenge us and redirect us accordingly, but as partners. It is imperative to forge partnerships, collaborations, and build relationships around assessment on your campus.
Because students often experience intersectionality of life experiences, forging these campus-wide connections allows us to more holistically support our students and thus, assessment becomes more culturally responsive. This two-way communication promotes transparency and feedback, as well as improved outcomes and assessment metrics for truly demonstrating the added value of student affairs. In addition, intentionality of focusing on the diverse life experiences of those we serve and work with only serves to enhance our work and bridge the gap between course learning and beyond-the-classroom experiences. It connects the assessment expertise from student affairs, while promoting and integrating the subject-matter expertise at the unit/department/program level.
Supporting the ‘whole’ student is critical in attempting to implement culturally responsive assessment. In student affairs, we can demonstrate how to discuss student learning and help our institutions best support our diverse students with our framework. It also provides specific perspective on the student experience regarding needs and issues affecting groups of students beyond the classroom.
In my office, we have worked hard to ensure that our assessment projects related to sense of belonging are holistically implemented via collaboration of Undergraduate Studies, International Affairs, and Enrollment Management. These partners measure sense of belonging early on in students’ academic journeys with the goal to connect students to the resources they may need. We meet regularly with our assessment teams within our division, advisory councils with faculty and staff, and our student advisory boards to ensure our assessment questions are aligned and designed to best measure ‘belonging.’ We also want to make sure our process for using the data is holistic and serving our students equitably.
Above is just one example how my institution is building culturally responsible assessment. Montenegro & Jankowski (2017) make some great points regarding culturally responsive assessment. I’d reinforce building an authentic culture of assessment beyond the classroom that is truly inclusive requires focusing on who we are serving, measurement and support of community to bridge the gap to focusing on students more holistically.
How are you building an authentic assessment culture? What does culturally responsive assessment look like on your campus?
Renee Delgado-Riley, University of Oregon