Reflection #3

eMM

Way back in weeks 9-10 of this course, we looked at Stephen Marshall’s e-Learning Maturity Model (eMM). It is a self-critiquing and reflection tool that can be used by tertiary institutions to evaluate their levels of maturity in five dimensions that categorise e-Learning. When assessing an organisation’s capability using eMM the practice is to select a number of actual courses (e.g. three or four). The selected courses should be be representative of organisational practice rather than exceptional. The purpose of eMM is to assess actual activity rather than intended activity for the future. A 4-point scale focuses on colours which give an overall picture (‘carpet’) of capability in a particular dimension, visually identifying areas for action and guiding an institution towards maturity.

I did an intuitive assessment of an online distance postgraduate course from a New Zealand university, focusing on just one process dimension (Delivery). The activity helped me to understand the comprehensive nature of the assessments, in this case the institution’s capability around the course’s learning outcomes, documentation and student support. I also created a mindmap to highlight the relationship between the different structural components of the eMM. I found that quite difficult to do, maybe because I’m a more linear thinker, or perhaps because I was already distracted by the looming prospect of the research article.

I do think this is a useful tool for tertiary institutions to use to self-critique their own e-Learning capability or maturity. I’ve mentioned several times that I’m on my institution’s Learning Technology Steering Group. On our social media site we have recently been discussing Marshall’s case study on a mid-size ITP with a lot of similar e-Learning goals to our own (Marshall, 2012). Interestingly, the ITP had been using the eMM for six years to benchmark its capability, with the biggest gains coming when they gave responsibility to the teaching staff and focused on developing staff capability and new teaching models. Rather than Roger’s model of innovation focused on early adopters, systems were put in place for adoption by all staff. That’s quite a commitment, requiring considerable resourcing to engage all staff.

Research assignment

I am struggling somewhat to clarify my research topic, finding it difficult to move from the review essay to a research focus. I have no primary research nor case study to analyse, which means I’m relying on limited secondary research on the topic I chose – innovating with online synchronous technologies to develop students’ academic literacies. My reading and thinking has moved back and forth from Niki’s ecological perspective, looking at all the stakeholders that affect and are affected by adopting the technology, to a focus primarily on the tutor and student. While web-conferencing provides a viable, flexible alternative to face-to-face academic support in individual consultations, the literature suggests real issues around group or class engagement when the tutor is not the regular class teacher and has little time or opportunity to build rapport and trust. I’ve been reading about Michael Moore’s Theory of Transactional Distance and wondering if that is a relevant concept to use to analyse online learning practices. Moore’s theory examines three factors that influence the teacher/student relationship in distance learning: dialogue, structure and learner autonomy (Falloon, 2011).

So, anyway, I digress. I’m not sure how many theoretical concepts to use. Is three too many? (I want to include the CBAM). What I’m thinking is that I need to turn this around and consider what guidance other tertiary learning advisors would appreciate if asked to teach and support distance students online. What information and skills do they need? What support do they need? How do they engage students in a meaningful and timely way? What are the barriers and pitfalls to look out for? And that’s where I’m at; purpose identified and the article still in the conceptual stage.

All that’s left to do is write it… time

References

Falloon, G. (2011). Making the connection: Moore’s theory of transactional distance and its relevance to the use of a virtual classroom in postgraduate online teacher education. Journal of Research on Technology in Education, 43(3), 187–209.

Marshall, S. (2012). E-learning and higher education: Understanding and supporting organisational change [Case study report]. Wellington, New Zealand: Ako Aotearoa National Centre for Tertiary Teaching Excellence.

Marshall, S. (2007). E-Learning Maturity Model: Process descriptions [draft report]. Retrieved from http://learn.canterbury.ac.nz/mod/page/view.php?id=186287

eMM Mindmap and Mini-Evaluation

The following is a visual representation of the eMM model and its structural components in relation to my context. Apologies for the difficulty in reading the detail – I’ll work on improving that.

eMM Mindmap

Strengths

The dimensions contain a number of process categories that examine performance based on examples of normal practice, so that the capability of the institution as a whole is examined, rather than just best practice. That means it looks at what the majority are doing, not just the star innovators.
The model aims to focus on standard, reproducable e-Learning processes, regardless of the different technologies and teaching practices in individual institutions, so that meaningful comparisons can be made across the educational sector.

Weaknesses

It is a very complex set of processes for an institution to follow. It involves the examination of large amounts of detail which requires considerable time and resource.
There is still an element of personal judgement in determining capability so the evidence that supports the assessment is critical.

Recommendation for using the eMM in my personal context.

Yes, I think that, with appropriate resourcing, the eMM would provide a comprehensive detailed assessment of the eLearning capability in my tertiary context. For an institution in which change is the only constant, it would provide a ‘big picture’ understanding of how well the institution is achieving its goals, how they shape up against comparable institutions, and how and where they need to improve.

Marshall, S. (2007). E-Learning Maturity Model: Process descriptions [draft report]. Retrieved from http://www.utdc.vuw.ac.nz/research/emm/documents/versiontwothree/20070620ProcessDescriptions.pdf

Victoria University. (2008). E-Learning Maturity Model: Version two [Website]. Retrieved September 15, 2013, from http://www.utdc.vuw.ac.nz/research/emm/VersionTwo.shtml

eMM Process Intuitive Assessment

I have chosen an online distance postgraduate course from a New Zealand university for this activity. I conducted an E-Learning Maturity Model (eMM) intuitive assessment of the processes L1, L2 and L3 restricted to the Delivery dimension. The university’s current eLearning approach is characterised by blended and distance learning approaches that provide flexible learning opportunities through a Learning Management System (LMS), particularly at postgraduate level.

Assessments of the Delivery dimension aim to determine how effectively the process outcomes are delivered within the institution. I have used the eMM capability assessment scale of: Not Adequate (NA); Partially Adequate (PA); Largely Adequate (LA); or Fully Adequate (FA).

The intuitive assessment results for each process in the category for the delivery dimension are:

L1. Learning objectives are apparent in the design and implementation of courses

LA – Formally stated learning objectives provided to a limited extent, either as narrative descriptions of the course outcomes or only in documentation
provided after enrolment.
LA – Most, but not all, assessments and learning activities contain explicit linkages to course learning objectives or restate learning objectives using different
wording.
LA – Most, but not all, assessments and learning activities contain explicit linkages to course learning objectives or restate learning objectives using different wording.
FA – Learning objectives formally and systematically address a range of student outcomes beyond the recall of information.
FA – Learning objectives are formally and systematically linked with course workload and assessment design and development.

Comments: The course has clearly defined learning outcomes, which are available online in pre-course documentation. The learning objectives refer to the assessment dimensions of Masters programmes. They emphasise critique and reflection, while the learning outcomes systematically refer to outcomes such as the ability to analyse, critically evaluate and apply.

L2. Students are provided with mechanisms for interaction with teaching staff and other students

FA – Interaction between staff and students provided formally through multiple complementary communication channels.
FA – Course documentation contains clear and consistently presented lists of teaching staff email addresses repeated in suitable places.
LA – Technical support is provided to students to assist them in making effective use of the available communication channels, but support is not actively promoted or provided to all students.

Comments: Interaction between tutors and students occurs in multiple mediums such as email, discussion forums and blog comments. Staff details appear on the LMS homepage, under course information and in list of participants. Technical support is available in terms of information on effective use of communication channels but support with technology issues is on an adhoc basis.

L3. Students are provided with e-learning skill development

FA – The relationships between all key course components and activities are conveyed to students formally and consistently.
LA – Formal opportunities for students to practice with e-learning technologies and pedagogies provided after commencement of courses, or only cover some technologies and pedagogies or some courses.
PA – E-learning skills support and training is provided informally and depends on the teaching staff skills and availability.
FA – Formal opportunities for feedback beyond the marks assigned for assessed work provided during all major course activities to all students.

Comments: There is a clear progression and link from teaching components to writing activities. Opportunities to practice with e-learning technologies are embedded formally in the course but support and training is provided informally online. There are formal opportunities to provide feedback on all activities via discussion forums and personal reflection.

Value judgement: The accuracy of the intuitive assessment is enhanced by clear and explicit documentation that is easily accessible on the LMS. However, the assessment is based on one course only and may not be representative of institutional practice. Some issues around accuracy may arise from a limited understanding of the characteristic statements for the Delivery dimension.

Disclaimer: This eMM assessment is based entirely on an intuitive assessment constructed on limited evidence and knowledge of the eLearning course in question, and completed purely as part of a formal course activity for EDEM630.

Reference:
Marshall, S. (2007). E-Learning Maturity Model: Process descriptions [draft report]. Retrieved from http://learn.canterbury.ac.nz/mod/page/view.php?id=186287