Introduction: Origins, Purpose, and Design of the Higher Education Assessment Syllabi Project
The syllabi project began in 2023 with a simple observation: despite the growing emphasis on assessment as a critical competency area for higher education and student affairs (HESA) professionals, the expected skills and knowledge around what assessment should look like in practice remains unclear. This is particularly felt for emerging professionals, who shared that they felt overwhelmed and confused when attempting to identify resources to help them on their assessment journey. Faculty described feeling similarly unsure of curriculum development, both in terms of what prior knowledge students might have and what could be considered the “core” topics of higher education assessment.
The purpose of this project is to provide a guidepost to HESA graduate students, faculty, and practitioners alike in identifying the trends across institutions teaching assessment coursework. Representatives from SAAL, the ACPA Commission for Assessment and Evaluation, and the NASPA Assessment, Evaluation, and Research Knowledge Community collaborated to collect and analyze over 100 syllabi provided to the research team. A team of ten researchers from the aforementioned associations divided into small groups to discuss the key themes, texts, and outcomes described in the syllabi, then met to discuss overarching trends. (For a more detailed review of our methods, please see the report published in March 2024.) We hope our analysis has created a resource for future faculty designing syllabi, graduate students wondering what to read next, and for assessment professionals seeking to critically evaluate the direction of the field and practitioner professional development needs.
This analysis is of course limited; many student affairs staff never attend a HESA graduate program, and syllabi illustrate just a small piece of overall course content. Syllabi are also living documents – they change by instructor, year, and even within a semester. This project should therefore not be read as what definitively should be taught or learned in HESA assessment programs, but as an attempt to clarify the core concepts and texts that have emerged within the field over decades of research, study, and practice.
Course Description
One area of analysis was focused on reviewing course descriptions in the syllabi in order to determine the primary focus of the course, the research methods being taught, and the professional standards and competencies being taught. Three reviewers conducted open coding and reached consensus on 14 emergent codes. The codes were reviewed by a fourth reviewer to ensure consistency in findings and alignment with research questions. The research team identified 7 course categories and themes from the syllabi, including:
-
Student affairs assessment
-
Higher education assessment
-
Research
-
Counseling based assessment
-
Program evaluation
-
Equity-minded assessment
-
Classroom assessment
The research team observed that most syllabi were tailored to higher education assessment (46, or 41.44%) and student affairs assessment (37, or 33.33%). Syllabi from the Southeast region were the most diverse categorically, yet also contained the highest frequency of syllabi focused on higher education assessment (12, or 26.09%) and on student affairs assessment (13, or 35.14%). The research team identified 3 research methods codes, including:
-
Quantitative
-
Qualitative
-
Mixed methods
Of all syllabi, 71 (63.96%) included a research method. Quantitative methods were most-mentioned (26, or 23.42%), followed by mixed methods (24, or 21.62%), and then followed by qualitative (21, or 18.92%). All but 5 syllabi either mentioned quantitative and qualitative together or mixed methods. In all 5 syllabi that only mentioned one research method, quantitative was the sole methodology mentioned. The research team identified 2 professional standards and competencies codes, including:
-
ACPA/NASPA Professional Competencies
-
CAS Standards
Of all syllabi, 41 (36.94%) mentioned professional standards or competencies. ACPA/NASPA Professional Competencies were most often mentioned (28, or 25.23%), with CAS Standards mentioned 13 times (11.71%).
Course Learning Outcomes
There were 98 syllabi containing student learning outcomes (SLOs) for their courses, 706 in total. A team of four reviewers coded SLOs using the 13 content areas of the ACPA ASK Standards, since these were created to specify the assessment competencies needed by student affairs professionals, regardless of role (ACPA, 2007). Each SLO was categorized into one standard as the primary theme of that outcome. Several themes emerged as frequent SLO topics not covered by the ASK Standards: overall purpose of assessment, theory/literature review, DEI in assessment, culture of assessment, and accreditation.
SLOs related to Assessment Design were the most prevalent. This is not surprising, since Assessment Design is an overarching outcome, including the “ability to articulate and execute an assessment plan at the program/service, unit, or division level (ACPA, 2007, p. 4)”. Effective Reporting and Use of Results and Selection of Data Collection and Management Methods were the next most prevalent categories, also not surprising as other essential competencies across assessment purposes.
Many of the other competencies are purpose and/or method specific, including Program Review and Evaluation, Articulate Learning and Development Outcomes, Assessment Instruments, Surveys used for Assessment Purposes, Interviews and Focus Groups used for Assessment Purposes, and Benchmarking. Several might be considered essential only for those going into roles dedicated to assessment, such as Assessment Education, Politics of Assessment, Culture of Assessment, and Accreditation.
Required Texts
This goal of analyzing required texts was to understand which texts were informing graduate education in higher education assessment. All syllabi collected were reviewed for any texts that were required for the course. Researchers reviewed publicly available information on course materials, publisher summaries, table of contents, and excerpts. Additionally, texts were coded for its relevancy to student affairs assessment, depth of content, inclusion of diverse assessment techniques, and use of pedagogical tools. Sixteen courses did not have any assigned texts, resulting in the final analysis including 68 assigned texts from 98 courses.
Texts required across three or more courses with distinct instructors
|
Citation |
# of Courses Requiring Text |
# of Courses Requiring Text |
|
Henning, G., & Roberts, D. (2024). Student Affairs Assessment: Theory to Practice. Routledge. |
34 |
35.7% |
|
Suskie, L. (2018). Assessing Student Learning: A Common Sense Guide. Wiley. |
14 |
14.3% |
|
Schuh, J.H., Biddix, J.P., Dean, L.A., & Kinzie, J. (2016). Assessment in Student Affairs. Wiley. |
13 |
13.3% |
Further analysis reveals that instructors use a core text to provide students with a foundational understanding of assessment and its different approaches, instructors then supplement their course with materials detailing assessment methods or issues in student affairs assessment. Additionally, 17 of the required texts dealt specifically with methods (seven focused on quantitative methods, four on qualitative methods, and four on general methods) and nine focused specifically on program evaluation. Required texts also differed depending on the specific graduate program or issues related to higher education assessment (e.g., classroom-based assessment for a general education curriculum).
Summary & Recommendations
This analysis helped clarify the state of assessment-related courses available in higher education graduate programs. Courses are often serving multiple purposes, not solely educating about assessment (e.g., a course about research and assessment or program evaluation and assessment). Despite assessment being an expected competency of student affairs professionals and standard across all functional areas, assessment-related courses are not necessarily required curricula in programs. Given the nature of assessment work, these courses should go beyond the how-to of assessment design and data collection to consider institutional context, political elements impacting assessment, and understanding philosophical roots to best guide practice. To be a successful assessment practitioner in different institutional contexts, an adaptable understanding of the various goals of assessment is needed.
A number of implications and recommendations emerged from our work. Emerging professionals should know there will be more to learn and experience about assessment beyond what they may have been exposed to in their academic preparation; relatedly, faculty should be sure to promote resources and further learning content for people to stay sharp about assessment going forward. Professional competencies, CAS standards, and assessment organizations can be resources to leverage and keep a finger on the pulse of the assessment landscape. Graduate program faculty and leadership can re-evaluate their existing course content for relevance, appropriateness, and clarity with respect to purpose, language, and practical application. While the field of assessment evolves and changes, responding to current learning contexts and needs, assessment-related courses should be revised to stay abreast of advancements in the field.
Link to full report: https://myacpa.org/higher-education-assessment-syllabi-project-initial-report/
Authors:
Tori Rehr, MEd; The Ohio State University; ACPA Commission for Assessment and Evaluation Chair
Natasha Jankowski, PhD; New England College; Assessment Expert Reviewer Representative
Paul Holliday-Millard, EdD; University of North Carolina, Charlotte; ACPA Commission for Assessment and Evaluation Chair Elect
Joe Levy, EdD; Excelsior University; Student Affairs Assessment Leaders Open Course Manager
Tanner Gill, MSW; University of North Carolina, Wilmington; ACPA Commission for Assessment and Evaluation Education Committee Chair
Shaun Boren, EdD; University of Florida; NASPA Assessment, Evaluation, and Research Knowledge Community Co-Chair
Shiloh Lovette, MA; Appalachian State University; Student Affairs Assessment Leaders Operations Coordinator
Resources:
Aaron, R. W., & Cogswell, C. A. (2022). The on ramp: Essential entry level experience and skills in student affairs assessment. New Directions for Student Services, 2022(178–179), 29–42. https://doi.org/10.1002/ss.20426
American College Personnel Association & National Association of Student Personnel Administrators. (2015). ACPA/ NASPA professional competency areas for student affairs practitioners (2nd ed.). https://www.naspa.org/images/uploads/main/ACPA_NASPA_Professional_Competencies_FINAL.pdf
Dean, L. A., & Langham, A. T. (2022). Graduate preparation in student affairs assessment. New Directions for Student Services, 2022(178–179), 43–51. https://doi.org/10.1002/ss.20427
Mitchell, A. A., & Dixon, K. M. (2022). Editors’ notes. New Directions for Student Services, 2022(178–179), 7–14. https://doi.org/10.1002/ss.20424
Saldaña, J. (2016). The coding manual for qualitative researchers (3rd ed.). Sage.