CLO Review Project - Online Video Training

The CLO Review project's face-to-face session logistical challenges required a pivot to an online format.  Yet designing effective videos for online learning is challenging.  To do this well, I employed a number of key good learning design practices included:

The video production principles: 

Along with chunking video (Harris et al, 2021), the video production principles (Guo et al, 2014) used:

•  Shorter videos are much more engaging.
• Videos that intersperse an instructor’s talking head with slides are more engaging than slides alone.
• Videos produced with a more personal feel could be more engaging than high-fidelity studio recordings.
• Khan-style tablet drawing tutorials are more engaging than PowerPoint slides or code screencasts.
• Even high quality pre-recorded classroom lectures are not as engaging when chopped up for a MOOC.
• Videos where instructors speak fairly fast and with high enthusiasm are more engaging.
• Students engage differently with lecture and tutorial videos.

The video design principles:  

From a content perspective, experience has taught me:

1. Paint a target first.  Show what is to be produced and how it's used.  This allows the learner to contextualise the topic and make personal connections to the learning.
2. Minimise terms and definitions.  Even though the audience is academic, the learning has an pragmatic objective, thus traditional academic scaffolding is unnecessary.
3. Minimalist design.  Less is 1000x more because it strengthens the signal to noise ratio, allowing learners to hone in on what is important.  AI image generation has become useful for creating complementary abstract, sketch-like images - which the target audience responds well to.
4. Practice what I preach: I ask academics to produce chunked videos — so should I! 

To connect experience with research, the literature states: Mayer's (2009) research identified nine principles for more effective multimedia learning:

1. Multimedia effect: Better transfer when a message contains words and pictures rather than words alone.
2. Spatial contiguity effect: Better transfer when printed words are placed near rather than far from corresponding pictures.
3. Temporal contiguity effect: Better transfer when corresponding narration and animation are presented simultaneously rather than successively.
4. Coherence effect: Better transfer when irrelevant words, pictures, and sounds are excluded rather than included.
5. Modality effect: Better transfer from animation and narration than from animation and on-screen text.
6. Redundancy effect: Better transfer from animation and narration than from animation, narration, and on-screen text.
7. Pretraining effect: Better transfer when training on components precedes rather than follows a message.
8. Signaling effect: Better transfer when narration is signaled rather than non-signaled.
9. Personalization principle: Better transfer when words are in conversational style rather than formal style.

So, let's get to the point and see this training and you decide on the effectiveness:

The CLO Review Training 

Part 1 The CLO Review Project

The CLO Review project is updating the Course Guides Part A’s Course Learning Outcomes (CLOs) and Assessment schedules. This training has five parts: (i) understanding the need for review, (ii) writing effective CLOs, (iii) checking existing CLOs, (iv) reviewing assessment schedules, and (v) making necessary changes.

The review is essential due to changes in Program Learning Outcome (PLO) mappings and new programs being developed. It’s essential that CLOs align with assessments, PLOs and the Australian Qualifications Framework (AQF). While it might seem administrative, well-crafted CLOs are vital for course design and help students articulate their skills to employers.

 

Part 2: Writing great CLOs

Victoria University AQF4-10 CLO Verbs

Focuses on creating great Course Learning Outcomes (CLOs). A good CLO involves three elements: an observable verb (e.g., "validate"), the content (e.g., "engineering designs"), and the context (e.g., "meet specified requirements").

Designing CLOs involves aligning with various standards like RMIT Graduate Capabilities, Engineers Australia Competencies, and the Australian Qualifications Framework (AQF). Constructive alignment is achieved by each standard level referencing keywords from the level above.

CLOs should be clear, reflecting the most critical skills students will acquire and apply. They are not assessments, lists of topics, or tasks. Instead, they describe key capabilities, succinctly expressed in 4-6 outcomes per course. CLOs should be measurable and align with PLOs.

The example given evolves from applying engineering teamwork processes to applying collaboration and communication strategies in advanced multidisciplinary engineering projects. This alignment process involves careful selection of verbs and content to ensure compliance with both university and industry standards.

Part 3: Checking a CLO

Addresses common pitfalls in writing Course Learning Outcomes (CLOs). CLOs are approved by a program manager or a Deputy Head of Department (L&T), focusing on alignment with Program Learning Outcomes (PLOs), the Australian Qualifications Framework (AQF), and assessment alignment. 

Eleven common mistakes include incorrect structure, ambiguity, overlap, and inappropriate task focus. CLOs should be outcome-based, use appropriate verbs, and align with the correct AQF level and PLO keywords.

The scope should fit within the recommended range of 4-6 CLOs. Misalignment between CLO verbs and assessment types must be avoided to ensure proper course quality. Continuous drafting and redrafting are crucial for effective CLOs that reflect the applied abilities students gain.

Part 4 Reviewing the Assessment Schedule

Covers reviewing the assessment schedule within your course guide, specifically Part A. An example structure includes the assessment identifier, type, weighting, and related CLOs (Course Learning Outcomes), with hurdle tasks noted in brackets. 

Consistency is between Course Guides is important so please avoid shorthand in CLO listings. To determine assessment types, refer to the RMIT SharePoint's assessment types database. The assessment type should align with the CLO action; for instance, a design task aligns better with a portfolio or report than with a quiz.

Simplicity in the assessment schedules ensures administration flexibility; detailed specifics belong in Part B. Consider CLOs' number for weighting judgments, but it's uncommon for assessments to cover all CLOs simultaneously. Such practices can trigger further investigation by TEQSA. Finally, any changes in your Course Guide should then be applied to your course.

Part 5 Making Changes

For support, contact the School Learning Designer.

The CLO Review project involves updating CLOs for 350 courses. Three strategies are employed to minimise impact: completing only the CLO and Assessment schedule, automating the review process with forms, and lastly, the Engineering L&T team offers suggested CLOs that can be adapted.

Communication will be via emails from RMIT University via SmartSheet, providing necessary information. The CLO Review team publish the final versions to the Course Guide.

After CLO changes are implemented, you might need to update assessments, rubrics, and learning activities before the course's next run. Immediate focus is on updating the CLO and assessment schedule. For assistance, contact the School of Engineering Learning and Teaching team via email. Your contributions enhance the engineering program for students. Start now to ensure timely updates.

References

Guo, P. J., Kim, J., & Rubin, R. (2014). How video production affects student engagement: An empirical study of MOOC videos. In Proceedings of the First ACM Conference on Learning @ Scale (pp. 41-50). ACM. https://doi.org/10.1145/2556325.2566239 

Harris, A., Meehan, K., Wicks, C., Baxter, S., Baughman, S., & Fritz, S. (2021). The impact of lecture chunking format on university student vigilance. Teaching of Psychology, 48(4), 308-315.https://doi.org/10.33902/JPSP.2021272429

Mayer, R.E. (2009). Multimedia Learning (2nd ed.). Cambridge University Press.