- Should we do more eLearning?
- How effective is eLearning compared to instructor-led training (ILT), peer-to-peer coaching, and other types of learning?
- How do we know if eLearning actually produces learning?
- If we don’t measure it, how will we know its effectiveness? How will we improve upon it?
First, What is the Goal?We refer to program evaluation before the start of a program as needs assessment. We can’t design, develop, and implement an impactful eLearning program without first understanding the objectives. Needs assessment involves all stakeholders, including management, front-line employees, and customers. By understanding the current situation and what behaviors we want to change or improve upon, we can design an appropriate eLearning program. Program evaluation methods in the needs assessment phase include observation, informal dialogue, formal interviews, and surveys. Instructional design personnel seek answers to questions such as:
- What new knowledge, skills, and/or behaviors do key stakeholders (employees, management, customers) need to meet business challenges?
- What are measurable results to help determine success of online learning program?
- How does the eLearning program align with other learning activities supporting the learning objectives?
How is the Process Going?After determining objectives of an eLearning program, the instructional design team gets to work designing and developing a program to meet program objectives. In-process evaluation, also known as formative evaluation, helps the instructional design team pre-test how well the program aligns with objectives. Formative program evaluation methods include a review of content and activities material, and may include a more formal product test, or prototype evaluation, before going “live.” Formative evaluation may be informal and conducted internal to the instructional design group. Instructional design team members ask questions such as:
- Does the design include all identified objectives?
- How might design elements encourage more engagement in the content?
- How will each element facilitate learning?
- How will we measure results?
What are the Results?After implementation of the eLearning program, we conduct summative evaluation to assess the impact of a program. The focus of summative evaluation is on the outcome. Summative program evaluation methods encompass formal assessments, often external to the instructional design organization. Summative program evaluation methods incorporate questionnaires, interviews, observations, and traditional end-of-course testing. Summative program evaluation is an integral part of program design. Common topics for a summative program evaluation include:
- Most helpful elements
- Most engaging elements
- Elements most pertinent to the participant’s job
- Elements participants expect to immediately apply on the job
- Suggestions for improvement