05/2016 - The systems thinking and training blog - Bryan Hopkins

Go to content

Main menu:

Designing better smile sheets: essential reading

Published by in Evaluation ·
I have just been reading a new book by Will Thalheimer called "Smile Sheets", and an excellent read it is too.

If you don't know Will, he specialises in reading academic research about learning and thinking about how this can contribute to training. Training is, of course, an area where there are various strange practices based on mythical facts. One of my favourites is the cone of experience, the claim that we remember 10% of what we read, 20% of what we hear and so on. Claims such as this are presented in training, and because they seem to make some sense get repeated, and slowly becomes fact. This particular topic is one that Will has discussed in the past, and I can recommend a visit to his blog to learn more (www.willatworklearning.com/).

Anyway, he focuses in his latest book on the smile sheet, or to give it its polite name, the reaction questionnaire (a la Kirkpatrick). Although this is the bedrock of most training evaluation activities, the book discusses in some detail the lack of research data to prove that it is meaningful in any way. This is because of a number of different factors. One is that the types of questions often included in reaction questionnaires are often poorly constructed from a statistical point of view, and force the learner into giving a positive response. Another is that surveys conducted in the training environment while the training is still under way are heavily influenced by the fact of being there and because there is no time for reflection on what the training has been about. Finally, there is very little evidence to show that merely reacting positively to a training activity means that there will be learning, which is a fundamental principle in the Kirkpatrick framework, which, of course, underlies much thinking in training evaluation.

Will then goes on to talk about what learning actually means and provides a practical guide to how to design 'smile sheets' which can actually produce meaningful and useful data. It is a most entertaining and illuminating read, and I certainly wish that I had read it before sending my own manuscripts to my publisher!

If you do get involved in any way with training evaluation, buy yourself a copy. At $25, it's well worth it.

Update on my forthcoming book

Published by in Evaluation ·
This is a slightly different entry, but is an update with information on my new book, "Learning and Performance: A Systemic Model for Analysing Needs and Evaluating Training ".

This is a practical guide for using systems thinking concepts such as boundary definitions, multiple perspectives and relationships in carrying out training needs analyses and programme evaluation.

It explains how to use techniques such as the Viable Systems Model and Soft Systems Methodology to explore areas of concern in organisational performance, in order to identify a holistic set of solutions which can improve performance. In the case of evaluating training, it uses these tools to provide a practical approach to evaluating both the learning and impact of training.

The book will be published by Routledge in late 2016 with an expected cover price of about £45. However, after some discussions with my publisher, we have decided to take pre-publication orders at the heavily discounted price of £20.00. That looks like a very good deal.

If you are interested, all you have to do to register a pre-publication order is click here to contact Jonathan at Routledge: jonathan.norman@tandf.co.uk.

Why I like 70:20:10

Published by in Informal learning ·
I have just finished reading the report "70+20+10=100: The Evidence Behind The Numbers", produced by Charles Jennings and Towards Maturity. A most interesting and worthwhile read.

For clarity, the 70:20:10 model refers to an observation that 70% of what people learn comes from real life and on-the-job experience, 20% from working with other people and 10% from formal training. These figures came from observations on leadership in a largely male target group, and, as the report acknowledges, different figures have been derived for female groups. Other research, not referred to in the report, discusses how 80% of what people learn comes from 'informal' means.

So there is some disagreement about the numbers, but this is largely because of the difficulties of actually defining what these categories of learning mean: what is the difference between 'on-the-job experience' and 'working with others'. What exactly is 'informal' learning?

But really, the numbers are not important. Why 70:20:10 is such a useful concept is that it provides a simple model around which people can conceptualise the importance of integrating formal and informal learning, which is something which the training industry has struggled with for many years. As the report says, its value is in helping people to realise that learning is a complex, multi-faceted activity, but that taking steps to facilitate non-formal learning opportunities and integrating them with formal training can bring rich rewards to organisations.

As I read the report I felt a strong sense of vindication that my own systems-based approach to analysing performance issues and developing training strategies is completely justified. Using a systems approach automatically means that we develop an understanding of each part of the 70:20:10 triad, about the dynamics of the workplace, how people work with each other and share information, what barriers and enablers may exist to implementing new knowledge and skills and so on. This can help us to design training which explicitly helps people to integrate their informal learning opportunities with training. Definitely a good thing!

Copyright 2015. All rights reserved.
Back to content | Back to main menu