Although the tradition of dressing in costumes and trick or treat are meant to be a fun activity for children and adults alike, the notion of celebrating things that are terrifying, menacing, or scary is the reason why I don’t particularly care for Halloween. Similarly, for those responsible for assessment, addressing the question of “so what” or “what’s next” might be a frightening experience once the data has been collected.  As institutions emphasize the need for continuous improvement for the purposes of student success, persistence, completion, and overall effectiveness, the actions required to close the loop (i.e. communication, sharing, making decisions) with assessment results is important.  However, the process of doing those things may be daunting for even the most experienced assessment pros. 

Continuous improvement is about making changes, large or small. Reporting results involves the communication and conversations necessary for ensuring change can or will occur.  As such, the assessment process requires both to work in unison. Therefore, what can make taking action for the purpose of continuous improvement ominous includes: (1) not fully understanding the implications of what the data means; (2) concerns about discussing results – with faculty/adjuncts teaching specific courses or colleagues who are involved in programs/initiatives – where a perceived difference in philosophy or direction may exist; or (3) the lack of institutional resources available for making the necessary improvements.  Addressing one or all of these challenges is key to ensure that the culture of assessment doesn’t become stagnated or stalled.   

http://clipart-library.com/clipart/1830942.htm

What’s “Lurking” Behind Those Assessment Outcomes? 

Have you ever had an instance when your assessment outcomes, student surveys or completion, persistence, or retention data provides a slightly different picture than you anticipated? 

Has that information left you feeling uneasy when institutional leaders expect you to provide an analysis and report the information (including follow-up actions) within a specific time frame? 

The indications of not meeting objectives and targets can paralyze faculty and administrators responsible for assessment.  Having to present findings can be a challenge if the data implies something other than what was initially anticipated or foreseen. If the results manage to leave individuals with a feeling of uneasiness, then developing an assessment action/improvement plan to close the loop might leave one paralyzed!

The key to facing what might be “lurking” behind your assessment outcomes just might be fostering a partnership with those who work in your Institutional Research (IR) department. Joining forces with these subject matter experts can be a guide to clarifying data needs and identifying exactly what should be addressed as part of the assessment. Additionally, these folks can help tackle: aligning the purpose with the data needed, making improvements to data collection, providing you/your team with insights for developing survey questions or simply revisiting data analysis.  Soliciting IRs assistance (i.e. inviting the IR Director to preliminary planning meeting to review methods of data collection or having IR review data and providing a summary analysis prior to finalizing for reports) in order to understand what your data means can be a small yet significant step towards closing the assessment loop (telling the story) thus removing the “heebie-jeebies” from continuous improvement.    

https://abstract.desktopnexus.com/wallpaper/2019637/

Having a “Spine-chilling” Discussion

Have you felt queasy about discussing your assessment results? 

Does the thought of reporting-out to different stakeholder groups (e.g. advisory committees, academic leadership, faculty, adjuncts) linked to your courses, programs or initiatives leave a “pit” in your stomach? 

With participant involvement being an important aspect of assessment, input from these various constituencies is a vital aspect of closing the loop. However, telling the story of the data and discussing what the findings mean or imply in order to address the need for continuous improvement can be intimidating.  Varied perceptions regarding what to do could lead to a course of action which is transformational or simply maintains the status quo.  Such discussions can lead to additional questions which may not be easily addressed or to startling discoveries not originally observed by the assessment professional.  Therefore, causing some angst if it’s decided that the action requires going down a path (towards change or improvement) that is uncomfortable, unimaginable, or seemingly impossible. 

However, navigating such a discussion doesn’t have to be terrifying. One of the benefits of having the assessment results conversation with such a diverse mix of institutional stakeholders is utilization of the feedback as an avenue for soliciting new ideas and differences in approach towards making the needed changes. Such conversations can be facilitated by using a SWOT (strengths, weaknesses, opportunities, and threats) analysis to capture insights from each constituent group, work with IR to categorize them by themes, and then further prioritize each theme into assessment action items.  Once actions have been developed, the assessment champion can call upon members from their respective constituent groups to assist with tracking and documenting to support ongoing communication as change process unfolds. 

A few examples that may assist with calling upon related members to aid in tracking and documentation supporting communication include:

• Using 30 minutes of a department/division meeting to facilitate mini assessment discussions throughout the term where ideas can be communicated or shared,
• Establishing an informal group of assessment peer reviewers who would be available to conduct preliminary reviews of assessment findings/results (as needed)
• Providing insights to faculty and staff prior to them submitting reports, or utilize assessment share fairs as a forum for exchanging ideas for intervention, further action or improvement.          

https://www.wtvy.com/content/news/Wiregrass-Trick-or-Treat-times-498573491.html

Tricks or Treats?

Have you ever outlined the action plan or continuous improvement idea only to wonder where the resources will come from? 

Have you ever prepared the justification for the “need” only to be told to wait? 

When results lead to communicating a need for additional resources, even the most experienced of assessment pro can become timid. Nothing is more frustrating than attempting to move forward with innovative, improvement ideas if resources may or are continuously held at bay. It’s an unsettling issue that is both unpredictable and resurfaces like the monster under the bed!  Although many are encouraged to communicate the need(s) as part of the corrective/improvement action, if the action requires additional institutional resources (human, technical, financial, etc.) the challenge of receiving them can make folks wonder if this portion of the assessment process will lead to receiving a “trick” (promise of resources) or a “treat” (something to assist with making implementation possible).     

To overcome such a manifestation, assessment pros can help alleviate such a haunting notion by brainstorming creative ways to support the resource need, as well as advocate for the resources needed by providing additional data to help inform or provide context for the reasons why a request is being made.  Consider identifying other opportunities for collaboration with other college departments.  Research the potential for obtaining external funds through state, federal or foundation grants.  Be bold and ask external partners from business/industry to donate equipment and/or services from corporations.  Building such strategies into the closing the loop discussion and continuous improvement action planning may prove to be most beneficial. 

And if the answer to the resource request is still, “no” or “wait until the next time,” it’s important to make sure there is evidence that exists which shows the work has been done.  Documented evidence which even outlines the “why” an action or implementation plan was placed on “hold” can still be a powerful tool for future use when operational/strategic plans shift funding is released or the timing for alignment is more appropriate. Assessment professionals can remain advocates and partners in promoting data-informed decision making. Even when efforts for change are ultimately unsuccessful, documentation can provide historical context and evidence to strengthen future efforts. 

https://www.logodesignlove.com/ghostbusters-logo

Final Thoughts about those Assessment Fear Factors

Remember how I mentioned that I don’t particularly care for Halloween?  Well… it still gives me the creeps!  This may also be true for those of you responsible for closing the assessment loop.  If this is the case, do what I had to do: make personal adjustments (e.g., changing practice, engaging in professional development) as best you can and seek out a buddy to help navigate potentially scary situations.  The notion of broaching what might be a spine-chilling data discussion or wondering if the resources requested will end up some sort of trickery may be uncomfortable, but it can be done.  You may not totally overcome your assessment fear factors, but there are those “brave souls” who exist that have and I bet they’re more than willing to assist when called upon.   

So... who ya gonna call?  Assessment busters!

 Because this post was written from my perspective and not intended to be exhaustive, we invite feedback from our readers! What other scary scenarios face those of you responsible for closing the loop in assessment? Can you identify additional ways to remove the “fright” out of the process?

Nanette Smith, Rhodes State College