Education Technology Services

Measuring the Educational Journey: Assessing Learning Outcomes in Education Initiatives

Education plays a vital role in shaping individuals' lives and fostering societal progress. Across the globe, numerous education initiatives and programs strive to provide quality learning experiences to learners of all ages. However, the question that often arises is, how effective are these educational endeavours in achieving their intended outcomes?

In the pursuit of educational excellence, assessing learning outcomes has emerged as a crucial aspect of evaluating the effectiveness of education initiatives. By measuring the educational journey, we can gain insights into the impact of these programs on learners' knowledge, skills, and overall growth.

Here’s an overview of the Kirkpatrick model of evaluation – a model of evaluation often deployed for assessing the effectiveness of educational programs.

Developed originally by Professor Donald Kirkpatrick of the University of Wisconsin, USA, in 1959, the model was substantially revised in 2016 and re-christened the “New World Kirkpatrick Model". The model consists of 4 levels of evaluation. The first two levels are performed at the time of the training programme and assess how successfully the training met its learning objectives. The subsequent two levels measure how effectively the training resulted in on-the-job improvement in performance and ultimately on the company’s bottom line.

Level 1—Reaction: this level measures the immediate response of the trainees to the programme and answers questions like:

• How clearly did the instructor convey the information and how engaging was the programme?
• How relevant did participants feel the programme was to their actual work?

This evaluation is mostly done through questionnaires, interviews, and direct observation of the programme.

Level 2—Learning: this level measures what the trainees learned.

• What did trainees learn from the programme based on pre-and post-training tests?
• With programmes that focus on practical skills, trainees have to demonstrate the new skills they have learned.

This evaluation is done through tests (paper and practical), self-evaluation questionnaires, and interviews.

These two levels are the easiest to evaluate and so are the ones most conducted by HR Departments. Modern software allows the use of online evaluation which not only speeds up the process but also makes it easier to analyse the data and present the results graphically using appropriate charts. Evaluation of these two levels is most useful for programme improvement through the feedback they provide.

The real challenge is the evaluation done at levels 3 and 4, which focus on the actual impact of the training on performance and productivity.

Level 3—Behaviour: This level measures the actual transfer of knowledge/skills/attitude back on-the-job; consequently, this evaluation is done weeks and months after the training is completed, maybe several times. This level measures:

• Are the trainees successfully implementing what they learned when performing their tasks?
• Is the change in behaviour/attitude transient or lasting?

Success at this level requires a process of ongoing support, reinforcement, and continued learning. This level is measured through observation, interviews, and reports by supervisors (some companies use 360-degree reviews), and by metrics that measure the specific objectives of the training; e.g., customer satisfaction, effectively closing sales, improved performance of tasks, fewer errors, etc.

Level 4—Results: This level is the most difficult and costly level to measure; hence, this level of evaluation is done mainly for the most critical and costly programmes.

The  “New World Kirkpatrick Model" stresses using the model from the end to the beginning. Starting with identifying the level 4 results you want to accomplish you specify ‘Leading Indicators’ that will show if you have achieved your goals (for example, depending on the objectives of the programme—increased employee retention, increased productivity, improved customer satisfaction scores, increased sales, reduction in errors/waste, improved quality, improved staff morale). Having established the desired level 4 results, you then determine the specific changes in behaviour required to achieve these results. It is important to operationalise these behavioural changes as specifically as possible so they can be measured. Having established the metrics, you determine the methods you will use to gather and analyse data. Finally, based on this analysis, you design and implement the training programme.

One cost-effective approach to measuring levels 3 and 4 involves the establishment of an online community restricted to the trainees and trainers of a recently concluded programme. This online platform allows participants to share the challenges they face in applying what they learnt; participants share their solutions and support each other. This approach uses the power of peer-to-peer learning along with positive reinforcement from trainers as the trainees engage in this process. This online community is kept active for several months and at the end a summary of what transpires is prepared as a level 3/4 evaluation report for managers.

By systematically evaluating the learning process, knowledge acquisition, behaviour change, and overall organisational impact, the Kirkpatrick Model helps organisations make data-driven decisions to improve training outcomes and maximise return on investment. This model and other such methods serve as a valuable tool for organisations seeking to optimise their training and development efforts.

* For organizations on the digital transformation journey, agility is key in responding to a rapidly changing technology and business landscape. Now more than ever, it is crucial to deliver and exceed on organizational expectations with a robust digital mindset backed by innovation. Enabling businesses to sense, learn, respond, and evolve like a living organism, will be imperative for business excellence going forward. A comprehensive, yet modular suite of services is doing exactly that. Equipping organizations with intuitive decision-making automatically at scale, actionable insights based on real-time solutions, anytime/anywhere experience, and in-depth data visibility across functions leading to hyper-productivity, Live Enterprise is building connected organizations that are innovating collaboratively for the future.