Time for the happy sheet to die!
“I will not be pushed, filed, indexed, briefed, debriefed or numbered.
My life is my own!” (The Prisoner)
We need to reconsider how learning is evaluated. In a world where we need to encourage collaboration, dialogue and better narrative we set out to determine participant/student satisfaction from learning/keynote/masterclass events via "happy sheets" (end of course reviews). It doesn't change when L&D professionals ask for training feedback in organisation: some Likert Scale questions - 5 points, "very satisfied" (strongly agree) through to "very dissatisfied" (strongly disagree) - and a couple of boxes for "open" responses that rarely contain anything meaningful "thoroughly enjoyed the course, thank you" through to "waste of my time!" The idea being that "feelings" can be expressed as a number, which contributes to big/ger data - I'll finish this blog with an explanation of why this is not a good thing.
What is the difference between a "very satisfied" and "satisfied?" Is it the same for everyone? What the heck does "neutral" mean? Someone attended a course/event and didn't "feel" anything, doesn't have an opinion, is that the person/people I should be really concerned about? What about the nature of the topic or whether the speaker/lecturer is attempting to disrupt established behaviours? Do the audience have the wherewithal (maturity) to reflect on whether the message/experience was good for them at that moment in time?
There will be those that argue that the data produces a "sense" of satisfaction. Okay. When was the survey taken? How does it correlate to impact - that magical change in behaviour that is the objective of adult learning? What about results - an increase in productivity or career development? How does feedback from one group/cohort relate to the next group/cohort who will receive the same message/experience; is it right that I take the opinions of one group and project their preferences onto another group that I haven't even met yet? What about the influence of timing, for example, politics and an audience operating in a volatile, unsafe, employment situation or students under exam pressure? How about the difference between support for a message/change and commitment to a message/change?
Some will say that anonymous "happy sheet" feedback provides authentic feedback. Really. What's the bloomin' difference between a "5" and a "4!" I'm deeply reflective and I want "5"s! What about context? A name can provide a setting for the deliverer, linking experience and performance to the message/content, which provides valuable context when the lecturer/trainer is reflecting upon their own performance.
How about those survey designers who use the dark arts to game feedback? Design a feedback sheet, ask a participant for three things they don't like about an event and then ask them to rate the event using Likert Scale questions; they will more than likely express a lower level of satisfaction than a feedback sheet that asks for ten things they don't like about the event, followed by Likert Scale rating questions.
Major decisions are often made as a result of such feedback. Surely, it is time for the "happy sheet" to die!
In the commercial world I take an authentic approach to my delivery and my subsequent request for feedback. During my presentation/keynote/masterclass I model authenticity via critical reflection. At the end of the event I let participants know that I will not be using a "happy sheet" and why. I then ask them to engage in meaningful dialogue: email me; phone me; meet me for a coffee; any format you like, aside form a five-point rating scale, but engage with me. I want to know if people feel compelled to engage, whether through a positive or negative experience. I want them to be able to express their opinion and for me to be able to understand the complex web of reasons that opinion.
Through such self-directed engagement I can better understand whether their feelings are directly related to me and my delivery or their context - a real world example: a person felt negative about the core message because they were under pressure at work and it seemed as if my message was linked to leadership setting the audience up for redundancies. I would never have understood this from an anonymous Likert driven "happy sheet" and the participant admitted that they would never have written their opinion on such a sheet. What about those who are dissatisfied, but don't feel compelled to engage - some might ask, how much do they care anyway? You can reach out to such people, pick up the phone or email, show them you care, and provide an opportunity for such engagement. You can only open the door, they have to walk through it.
We need to create better narratives. Big Data is reportedly set to rule the world and that worries me - rubbish in/rubbish out. In the case of Universities, excellent institutions are under pressure to improve "satisfaction" and therefore are driven by such numbers. Why? Numbers don't tell us why someone "feels" a certain way and surely, in such a hyper-competitive, hyper-complex, marketplace we need to understand such things. For centuries scientists have been attempting to catalogue people using set of numbers or letters. It seems that such efforts are increasing in intensity, the more powerful technology the greater the conviction that people conform to a behaviour pattern or can be reduced to a number. I'm not convinced that we are close to achieving this. Until such a time, please, consider killing the happy sheet.
For those that will argue the Phillips ROI Model - tell me how many organisations, including Universities, move beyond the numerical analysis of "feelings" to analyse results, impact and ROI? The answer? Very few!
