Menu

Slay Your Campus Assessment Monster

Slay Your Campus Assessment Monster

Is your assessment house the kind that gives away the full-sized candy bars or one that gets egged every year, as is tradition, by the neighborhood kids?  With a keen eye and some patience, you can turn into a regular Assessment Van Helsing, and slay your campus assessment monster.  Assessment monsters can be policies, procedures, traditions, or attitudes that lurk in the shadows of your assessment culture, and seek to undermine or destroy the work you are doing to make assessment a positive word on your campus.  Here are a few examples of monsters you might see on campus, and some tips on how to stop them in their tracks:

Frankenstein

Frankenstein old movie actor in black and white photo

 

“It was on a dreary night of November, that I beheld the accomplishment of my toils. With an anxiety that almost amounted to agony, I collected the instruments of life around me, that I might infuse a spark of being into the lifeless thing that lay at my feet.”
– Mary Shelley, Frankenstein

 

It’s alive!  Alive! (But just barely.)   Is assessment on your campus a hodge-podge of miscellaneous ideas and practices?  Does the right hand not understand what the left hand is doing?  Perhaps, assessment, like Frankenstein, is just misunderstood. If your assessment culture is a Frankenstein, you’ll want to identify the best parts of your culture (perhaps focusing on a particular department’s collection/use of data or a general strategy) and find ways to spread those practices to the rest of the body. Look to develop a cohesive divisional assessment plan inspired by the best practices within your division.  You could also infuse your practice with a jolt of electricity by reframing assessment and developing an assessment philosophy that encourages inquiry and curiosity over compliance.  Look at examples like the University of Tennessee’s new Office of Assessment and Strategic Initiatives for Student Life for how this new department frames the work that they do and provides a common space for the dissemination of resources so all the different parts of their organization are working under the same common language and philosophy.

Invisible Man

Invisible man old movie actor in black and white photo

“For the man who stood there shouting some incoherent explanation, was a solid gesticulating figure up to the coat-collar of him, and then — nothingness, no visible thing at all!”
― H.G. Wells, The Invisible Man

One of the great tragedies of the Invisible Man was his great power came with the price of him never being recognized for the work that he did.  While this often leads him to insane rampages in the various incarnations of his story, this monster’s presence on your campus is not so severe.  Many departments are doing great work to inform their practices with the data they collect, but do not have the means to communicate those efforts to a larger audience.  Rather than keeping them invisible, work to establish a divisional effort to share assessment data internally and externally.  I like to start with students so they know that the information we constantly ask for is being put to good use. The University of Vermont’s “You Said, We Acted” campaign is one of many that tell a cohesive story to students on how data turns into change on campus, and one of the many ways you can make data more visible to your students.

Werewolf

Werewolf old movie actor in black and white photo

“There are such ships, there are such logs in the swamps of our minds, and they rise to the surface of our thoughts for a moment, only to sink again. There are such ships sunk in the wastes of our lives. The years have washed over them. They are forgotten. And yet they rise, ghosts of a past that is ended. They float before us for a while to our own great astonishment, then they settle down again and are as if they never existed.”
― Guy Endore, The Werewolf of Paris

What is that in the sky?  Why do I feel this change bubbling up inside me? Is there a full moon rising?  Nope, just Middle States!  Assessment monsters often pop up when you least expect them, but an assessment werewolf shows up right around accreditation time and disappears into the woods soon after the report is submitted.  They are vicious beasts that consume data and are unhinged by the deadline of this single report.  I, too, am very motivated and driven by deadlines, but the appetite for information when accreditation season comes can often be overwhelming.  One suggestion might be to work off of this deadline motivation and build a self-study framework or an annual reporting plan that will set major reporting points that are staggered between accreditation cycles.  A good example of this is the reporting structure developed at the University of Houston.  Feed this assessment beast periodically, and you’ll have enough data to keep the werewolf at bay when that SACSCOC moon rises in the sky.

Dracula

Dracula old movie actor in black and white photo

 

“Truly there is no such thing as finality.”
― Bram Stoker, Dracula

 

While those of us who lead assessment on our campuses struggle to get many of our colleagues on board with integrating assessment into their work, there are some departments that once they get a taste for data, they can’t control themselves.  They want more, and have fully embraced the idea of assessment as a cycle that never ends.  They’ll assess anything and everything to fill the bottomless pit of information that drives them.  We, unfortunately, have created this monster ourselves.  Rather than drive a stake through the heart of this one, though, we need to refocus their drive to become more intentional.  Here, I would suggest developing departmental assessment plans that span multiple years and are tied back to specific goals for that unit.  Look to Iowa State University’s Assessment Plans for inspiration here.  Some tough love might be in store for you assessment Dracula, as well.  They’ll have to stop doing some things for the sake of the greater good.  With a few tweaks, though, they can still embrace their enthusiasm for data while making sure assessment isn’t sucking the life out of their work.

Mummy

Mummy old movie actor in black and white photo

“Burn the scroll, man. Burn it! It was through you this horror came into existence.”
– John L. Balderson, The Mummy

Has your assessment culture fallen victim to a centuries-old curse?  Probably not.  Perhaps, though, you are cursed with vestiges of an old assessment culture that refuses to rest in peace.  Before that culture comes back to life to destroy you as prophesized in some ancient mythology, bring that mummy into the 21st century by updating its linens and wrapping it in a new framework.  This could be a new policy or initiative that spans multiple departments and drives the assessment culture forward to a common goal instead of being stuck in the past. A good example of this comes from work we are doing here at the University at Buffalo to redefine the student employment experience through the NACE competency framework.  We have not completely changed the nature of the work done, but used this as a lens to focus the acquisition of certain skills via on-campus student employment. It has re-energized a lot of our departments to devote time to evaluate and make this experience more intentional for students without the hassle of a more drastic shift in how we provide these opportunities.

Like Freddy Krueger or Michael Myers, these monsters will come back for sequels, and while some of the details will be different, the story generally stays the same.  Might it rise up in a different department?  With a change in leadership? No one knows for sure. Just know you have the weapons at hand through SAAL and your colleagues to slay your campus assessment monster. 

What other assessment monsters lurk in the depths of your departments? Use the comments below to respond let SAAL know. 

Happy Halloween!


Daniel Kaczmarek, University at Buffalo 

Go Back

Although I'm reading the blog a little late I really appreciate the analogies used and the mechanisms and resources suggested for conquering them. This was a very creative way to explain and digest assessment that everyone across the campus can relate to. I definitely plan to share. Thanks!

Reply

What a creative and engaging description of the monsters we face in our assessment work! I will echo the praise from others already posted here. Kudos! Thank you for contributing this piece to the SAAL blog.

Reply

Fantastic Daniel! I see our category clearly. Thank you for not only the insights but the resources. This article is truly helpful. I'm sharing it with our assessment team at the November meeting.

Reply

Thanks for the kind words, Sharon. Hope the assessment team can benefit as well. Happy Assessing!

Reply

Excellent piece! I appreciate the time you took to weave the stories of these monsters into types of assessment challenges and providing example institutions, no less! Very entertaining! I think another assessment monster would be Mr. Hyde. This could be seen as two different assessment challenges. One challenge is those individuals that we consult with about how to improve the quality of their records by choosing different measures. They are engaged and motivated during the meeting and even take good notes, but, in the end, they just copy/paste last cycle's measures into this year's plans at the last minute without any of the changes that were brainstormed together. Quote from the book: “He is not easy to describe. There is something wrong with his appearance; something displeasing, something downright detestable. I never saw a man I so disliked, and yet I scarce know why. He must be deformed somewhere; he gives a strong feeling of deformity, although I couldn’t specify the point. He’s an extraordinary-looking man, and yet I really can name nothing out of the way. No, sir; I can make no hand of it; I can’t describe him. And it’s not want of memory; for I declare I can see him this moment.” A solution to this would be team consultations so that everyone in the department is on the same page, hold themselves and one another accountable, and share the same concerns and justification for chosen strategies. Another Mr. Hyde could be external benchmarks like national averages or other accrediting agency requirements. Colleagues sometimes think these benchmarks are what should be attempted to be met; however, they are sometimes harmful to assessment and improvement efforts based on the institution's very unique culture, resources, student demographics, etc. SACSCOC Assessment Principles do not dictate any specifics, really. They want institutions to be creative to drive (and document) their own efforts towards internal improvement. Quote from the book: "It was on the moral side, and in my own person, that I learned to recognize the thorough and primitive duality of man; I saw that, of the two natures that contended in the field of my consciousness, even if I could rightly be said to be either, it was only because I was radically both; and from an early date . . . I had learned to dwell with pleasure, as a beloved daydream, on the thought of the separation of these elements." An example of this is benchmarks for programs such as Upward Bound and TRIO. Our colleagues may see that it is less work to use the same measures for both funding requirements and SACSCOC assessment; however, the funding measures are determined by data about a cohort that was established four years ago. A lot can change in 4 years, and this measure does not fit well into common annual or every other cycle assessments that SACSCOC recommends. Also, any efforts towards improvement is only a hunch that may have worked four years ago but would be a misjudgment today. I had a lot of fun analyzing our assessment monsters and connecting it to a story. This kind of practice flexes our assessment brain muscles :) Metaphors are truly powerful! Quotes by author Robert Louis Stevenson and many thanks to Sparknotes.com.

Reply

Zaneta, I hear you loud and clear about external benchmarks. They do have value, but only if an institution has a good understanding of themselves and their unique culture. Setting arbitrary goals based off of a cohort, without that understanding of self, can be destructive. Great addition to the litany of monsters.

Reply

I concur with Wallace. I grew up watching these movies on Saturday afternoons (way before cable). This is a great way to tell the story of assessment issues many folks struggle with. Well-done!

Reply

Thanks, Gavin. Greatly appreciated.

Reply

Wow, I loved this essay! Thank you. I love a good story (and movie) and enjoyed the away Daniel used horror classics to elucidate the challenges and opportunities inherent in assessment. I plan to use the resources to help colleagues build a culture of assessment and evidence in my student affairs organization. Our Strategic Planning and Assessment (SPA) Committee has endorsed a proposed framework for assessment which will provide the foundation for our work.

Reply

Hi Wallace,

Thanks. I'd love to see that framework or hear about how that committee developed it. Those kinds of stories are so beneficial. You should consider sharing perhaps in a guest blog or other means with SAAL.

Reply


Comment

Blog Search

Comments