The piggy back ride
Let me continue from where I have stopped last time; the MDP project. As promised, this post is about some un-resolved pedagogical issues related to the above module. As admitted in my previous post, despite its recent success, MDP (level 5) is not without its pedagogical issues. Even though not as much as we used to get, we do still receive complains from students about various aspects of the module. Most of the complains are about the way the students are assessed in their groups.
Undoubtedly group learning delivers major benefits in higher education teaching and learning. Not only it can be used to encourage deeper learning, but it can also be used to promote student autonomy (Freeman 1995). This is often achieved by transferring some of the responsibility for teaching and learning to students. This includes elements such as peer tutoring, group discussions, group assignments and peer assessments. In particular group assignments can be a useful strategy to reduce academic time in feedback and marking.
In total more than 360 students take part in the MDP (level 5) module every year. Usually, the module contains 4 assessment points. Assessing and giving feedback for 1440 pieces of work is not a practical option for the academic staff members involved within the module, especially when the university policy of 3 weeks turn-around time for assessments and feedbacks is to be adhered to. Due to this reason (among others, such as the intended learning outcomes and module specifications), group assignments and peer assessments are unavoidable for this module.
Accordingly, the assessment framework for this module is as follows:
- An individual project execution plan for 20% of the module mark assessed by the tutor and individual written feedback given to individual students
- An interim group presentation for 20% of the module mark, assessed by tutors on group basis, but the marks are adjusted for individual members based on an effort log scoring system by peers.
- The final group presentation for 30% of the module mark assessment details similar to the item 2 above.
- Individual reflective commentary about the individual and group contributions to the project for 30% of the module mark assessment details similar to the item 1 above.
As with most of the group based assignments, MDP (level 5) suffers mainly from two main issues. First issue is the non performing students effectively having a “piggy back ride” on the other members of the group whereas the second issue being, top performing students are concerned about their marks being affected by the performance of other members of the group. Both these issues are neither new nor unique to the module concerned but rather are common to most group based assignments. Brooks and Ammons (2003) detail the issue of “free riders” in group based assignments. They observe the links between the time, frequency and the clarity of assessment criteria and the level of “free riding” in group assignments. Indeed we have also noted a strong link between the timing of the assessment and the level of complains about “free riding”. When the assessments are due towards the end of the module, tendency towards not participating fully within group works is high (especially below average students) compared to when there are either continuous assessments or early module assessments. The logical explanation for this behaviour could be the way that students prioritise their academic work. It has been noted that students prioritise individual assessments over group assessments in addition to the normal work prioritisation according to the appropriate deadline. Having observed this behaviour, we introduced the interim group presentation to the module, so that students are continuously engaged with working in groups, minimising the tendency for piggy back rides.
Free rides or piggy back rides are not the only issue in group assessments. The flip side of the coin also poses a common complain, good students getting penalised for other group members’ activities. This has also been a main area of research in the recent past. For example, Conway, Kember et al. (1993) describe the same issue and suggest individualised making approach to group assignment as a solution. The depth of this issue is very significant as the ability of graduating with a first class for a good student may well be affected based on the simple reason that he has been assigned to a non-performing or poor performing group in a group assessment.
Lejk and Wyvill (2001) also view individualised making approach to group assignments as a good solution to overcome both the problems presented above. Indeed this seems a logical approach from many angles, provided that it does not create an overburden to the academic assessors. For the MDP project, a similar approach was adopted, where student will receive their individual mark for all the assessment components including group presentations. In this case all the group presentations are assessed by tutors as a group work, but the group mark is then adjusted for individual contributions of each member of the team. This distribution is based on a peer evaluation matrix where each team members agree on a score for each of the members.
The above described approach is based on various research outcomes. Various authors have developed different tools and mechanisms to facilitate the group marks individualisation process with varying degrees of success (see: Goldfinch and Raeside 1990; Goldfinch 1994). Within the lifespan of the new MDP delivery, we have adopted two such systems, one being a secret scoring system and the second being an agreed scoring system. Both these systems have their own advantages and disadvantages. As Lejk and Wyvill (2001) detail, the secret scoring system tends to be more reflective of the actual contributions, whereas the latter can be significantly influenced by the majority decision. While we will continue to address pedagogical issues of the MDP group assessment aspect, we will not be able to make all the students 100% satisfied, as students carry different perceptions about group assessments. However, listing to the students and their concerns, mentoring them and support them to overcome their issues with group work would undoubtedly help to better their perception about group assessments. Within the MDP module we allow this to take place on continuous basis by allocating personal tutors for each of the teams. In fact, by doing that we are ensuring we follow the UKPSF guidelines related to the provision of student support and guidance activities (A4). In addition, we also continuously seek innovative way of improving our assessment and feedback process within the MDP module addressing the “assess and give feedback to learners (A3)” aspect of the UKPSF framework.
As described above, my follow up activities would mainly be based upon removing pedagogical limitations of the MDP project.
[piggy back ride image from: http://resources1.news.com.au/%5D
References
Brooks, C. M. and J. L. Ammons (2003). “Free riding in group projects and the effects of timing, frequency, and specificity of criteria in peer assessments.” The Journal of Education for Business 78(5): 268-272.
Conway, R., D. Kember, et al. (1993). “Peer Assessment of an Individual‘s Contribution to a Group Project.” Assessment & Evaluation in Higher Education 18(1): 45-56.
Freeman, M. (1995). “Peer assessment by groups of group work.” Assessment & Evaluation in Higher Education 20(3): 289-300.
Goldfinch, J. (1994). “Further developments in peer assessment of group projects.” Assessment & Evaluation in Higher Education 19(1): 29-35.
Goldfinch, J. and R. Raeside (1990). “Development of a peer assessment technique for obtaining individual marks on a group project.” Assessment and evaluation in Higher Education 15(3): 210-231.
Lejk, M. and M. Wyvill (2001). “The Effect of the Inclusion of Selfassessment with Peer Assessment of Contributions to a Group Project: A quantitative study of secret and agreed assessments.” Assessment & Evaluation in Higher Education 26(6): 551-561.
Two birds with one stone
This post would be little different to my other posts. This time, my intention is not to project deep reflections about my educational practices (forgive me, this means minimal reference to literature, and I know, I was told by the PGCAP team I should not be doing this) but to tell a simple, yet inspirational storey.
Some may see higher education as a community service. I can’t fault them as higher education is all about building future communities. However, engage actively in community benefit programmes as a part of higher education curriculum delivery is not something we see every day. This is a storey about one such occurrence.
It all started about three years ago when the Member of Parliament (then) for Salford approached the School of the Built Environment (SoBE) at university of Salford, seeking expertise to help a local football club. The clubhouse of this football club was destroyed in a fire. The club was not in position (financially) to rebuild its clubhouse, without which they were facing the possibility of not being able to play in their league. Club was looking for community support to regain their former glory. SoBE’s assistance was requested especially to get a new design developed for the clubhouse.
At the time, we (myself and two other academic staff members) were developing a strategy to better deliver a project based module named “multi-disciplinary project” (MDP). This module was intended to provide the students the environment to allow working in multi-disciplinary groups mimicking the multi-disciplinary nature of construction projects. One of the main learning outcomes was to teach the students to appreciate each other’s work in a team environment. Being a group based project this module was facing its fair share of all the usual pedagogical problems. These problems ranged from complains about “free riders” (see: Brooks and Ammons 2003) to high performers marks being affected by “sloggers” in the group (see: Conway, Kember et al. 1993). Students were demoralised and we needed a new strategy to cheer them up and to get their buy-in for this module.
We saw the challenge presented to us by the Salford MP (then) as an opportunity and decided to kill two birds with one stone. We set the challenge to our students taking the MDP (level 5) module to design a new club house for the local football club. With this, we were trying to expose our students to a real word challenge and at the same time to help the local community. We wanted the students to feel the real sense of their work, we even organised a competition among all the groups (about 30 groups, 12 members in a group), and the design of the winning group was go ahead with the actual construction. This means, students in that particular group actually can claim the clubhouse standing proudly in the local vicinity to be their own real design!!!
We managed to achieve what we were trying to do with this module; students’ engagement was at its level best. However, I must admit that it was a risky strategy; it could have gone either way. Add to that, all the usual problems of group work and group assessments were still there. Even for a minute I don’t claim that we solved all the group work and group assessment problems through this approach. But, we felt the students were less concerned about these issues and they engaged actively and enjoyed the module. Not only the students, but also the local media and the local community were enthusiastic about the whole project and were following the developments closely. One of several articles about the project as presented by local media is available here.
Students did a marvellous job. They have received very high regards from various professionals involved in the project for the level of professionalism they have demonstrated. One student even got a placement offer from one of the organisations involved, based on his project performance. The selected design has recently got the planning permission clearance and on its way to being built!!!
We are now into the third year of this new mode of delivery. We must have been doing something good; for the last three years, we have had more than 7 requests from various other local organisations to take their community project as the basis for our module. We have so far helped a local football club, a local cricket club and a local pub through this module.
The moral of the storey is, like in any other business, it may pay to take calculated risks to be innovative with module design and delivery in higher education teaching and learning. To say the least, with the above we were following the UKPSF guidelines on innovative module design and curriculum development, and identifying the importance of using appropriate methods for teaching in the subject area UKPSF (K2). In addition we were addressing the UKPSK guidelines on design and plan learning activities and/or programmes of study (A1) when planning our activities within this module.
Lined to my previous note, still there are pedagogical issues to be addressed within this new approach. One such issue is the use of peer assessments effectively in group work. I will continue study these issues deeply and my next blog post will be dedicated to reflect upon this issue deeply. Onwards and upwards!!!
References
Brooks, C. M. and J. L. Ammons (2003). “Free riding in group projects and the effects of timing, frequency, and specificity of criteria in peer assessments.” The Journal of Education for Business 78(5): 268-272.
Conway, R., D. Kember, et al. (1993). “Peer Assessment of an Individual‘s Contribution to a Group Project.” Assessment & Evaluation in Higher Education 18(1): 45-56.
How “virtual” the teaching and learning can be?
Being “virtual” is something that I dreamed about for a long time. How fascinating could it be to overcome the barriers of geographical distance and hence to a certain extent, the time!
In my academic life, my research interests are centered around this very concept of “virtual environments”. I have been researching into virtual organisations, virtual collaborative environments, virtual research environments and of course, virtual learning environments for teaching and learning.
Virtual learning and teaching covers a wide spectrum of concepts, technologies and processes. The term virtual learning is often used synonymously with distance learning. Noting the above, Carty (1999) defines distance learning to cover education settings that deliver training and information between two or more places, utilising modes such as synchronous (using same time communications), asynchronous (communications that do not require participants to exchange information at the same time), one way (information delivered from one point to one or many other points), two-way (any communication in which the flow is bi-directional but not limited to synchronous), multi-point (information delivered simultaneously from one place to many other places) and multi-cast (usually consisting of transmission of a video or audio clip to the computers of many users).
Virtual (or distance) learning concept (hereafter referred to as DL) immerged in late 70’s mainly with the establishment of the Open University in UK and with the development of a mixed media approach to teaching (Hellman 2003). The success of DL attributes to the growth of the technological developments related to the Internet, increased awareness, proficiency and wide accessibility to the Internet (Lindner, 1999). Furthermore, the ability of DL programmes to provide skills development opportunities required for professionals to upgrade their skills in a more flexible manner while carrying out fulltime employment (Gammie et al., 2002) undoubtedly helped establishing its roots in vocationally oriented educational programmes (as the case with the quantity surveying education where my experiences and interests are based).
Needless to say, traditional teaching methods and practices may not be applicable directly within DL settings. When I first attempted to deliver an online synchronous lecture (for a cohort of 35 Masters students from all over the world), the first difference I noticed as a tutor was the sense of “isolation” or the “detachment” from the students. When conducting face to face lectures I often read students’ reactions and adjust the lecturing style (and sometimes the content to a certain degree) to get them more engaged. I believe this follows the UKPSF guideline (V1) which states respect for learners. However, I felt senseless when I was doing a lecture to a dumb computer for the first time, as I could not capture the reactions of the learners.
I firmly believe, distance learning or otherwise, all the learners should be given the opportunity for the best possible learning experience. This include, what I believe as the “social aspects” of learning. These often include aspects such as team working, peer learning, student-teacher interaction beyond the structured lesson, etc. Indeed, I am not alone bearing this view, this requirement has been emphasized by other researchers as well. For example, Whatley (2004) emphasize the same view with the use of Kolb’s (Kolb, 1984) stages of experiential learning (concrete experience, reflective observation, conceptualisation and active experimentation).
Commenting about the teamwork in DL settings, Whatley (2004:55) suggests,
“Students undertaking online courses should be given a similar opportunity (Kolb’s stages of experiential learning) to experience team working, but where face-to-face contact is not possible, technologies may be able to provide additional resources to make the online experience comparable”
Whatley’s comment above exposes a relatively unexplored area of educational research. To what extent the technologies used in DL cater for the “social aspects” of learning and how that aspect can be improved? Understanding this aspect is a core knowledge requirement in the UKPSF framework (The use and value of appropriate learning technologies K4).
In fact I believe the DL practices today are technology driven (practices are developed to suit the capabilities of the technologies available) rather than the technologies were developed to cater for the needs of DL. I believe more research would be needed covering this aspect, if the true potential of DL is to be explored.
I am now into the fourth year of my online lecturing (predominantly to masters’ level students), and I believe I am now a bit more matured. I do not get the “senseless” feeling that strongly now, and I can sense the students’ reactions better. For this I use the prompts more frequently than in face-to-face sessions, the prompts ranging from asking the simple question “do you understand?” to asking more direct questions from students (often pin pointed). In a DL setting, I realised, asking open questions on voluntary basis would not work as expected, as there are too many “hiding places” available. Further, I often custom develop the DL lecture material to include more self-phased learning material as exercises and often remain online for 15 minutes after the lecture session just to allow the moment of tutor-student interaction to go beyond the planned lesson. I believe these are in line with the UKPSF guidelines related to design and plan programme of study (UKPSF-A1) and development of appropriate subject material( UKPSF-K1). Further linking my research to teaching practices in well in line with UKPSF guidelines on “engage in continuing professional development in subjects/disciplines and their pedagogy, incorporating research, scholarship and the evaluation of professional practices (A6)”.
Looking back about the progress I made so in this particular area, I believe, there are further opportunities for improvement. Especially, with the level of confidence gained through various discussions we had in PGCAP module I believe, I can experiment with different student motivation and engagement techniques. One interesting experiment would be to use “props” in DL environments and my current lesson planning for the next semester involves devising strategy for using “virtual” props for teaching!!!
[cartoon from: http://neslihandurmusoglu.edublogs.org/files/2011/02/cartoons-yf6mps.png]
References
Carty, W. (1999), “Distance education in the developing world”, The advising quarterly for professionals in international education, Summer 1999.
Gammie, E., Gammie, B. & Duncan, F. (2002) Operating a distance learning module within an undergraduate work placement: some reflections. Education and Training, 44 (1) 11-22.
Hellman, J.A. (2003), “The riddle of distance education: promise, problems and applications for development”, Technology, business and society programme paper series, No. 9, United Nations Research Institute for Social Development.
Kolb, D. (1984), “Experiential learning”, London: Prentice Hall
Lindner, J.R. (1999), “Usage and impact of the Internet for Appalachian chambers of commerce”, Journal of applied communications, Vol. 83.1, pp 42-52.
Whatley, J. and Bell, F. (2003), “Discussion across borders: benefits for collaborative learning”, Education media international, Vol 40.1, pp 139-152.
To prop or not to prop…. is the question
Week 4’s theme was mainly about using teaching aids (well, that’s props for some of you and me). During the session, we all had to try and use an object we brought from home as a teaching aid for a mini teaching demo in front of the class. I must say, I am pleasantly surprised to see how effective some of the small objects can be as teaching aids.
Making the lecture “interesting” is one of the yardsticks to measure “good teaching” in higher education. The importance of this aspect is well noted within the UKPSF famework (The implications of quality assurance and quality enhancement for academic and professional practice with a particular focus on teaching – K6)
Reid and Johnston (1999) validate the above claim while highlighting the fact that, both teachers and students recognise “interest” as one of the main dimensions (1 out of 6) of good teaching in higher education. Within the same study, they have classified “using teaching aids” as one aspect of making the teaching interesting. However, in that article, they have not made it very clear how teaching aids can make teaching in higher education “interesting”. This is the focus of my blog this week.
Guilty as charged, I do not use teaching aids that often in my lectures (in fact, I can’t remember the last time I used a teaching aid in any of my lectures!). Does that make my lectures “boring” and am I a “bad lecturer?” What are these teaching aids anyway? These are some of the questions came to my mind at the beginning of last week’s PGCAP session with Neil.
Having seen the “mini teaching demos” on using teaching aids, I believe my lectures actually can be made more lively and less monotonous. Is that in synonymous with making them “interesting”?
As mentioned above, I strongly agree with the view that appropriate use of teaching aids has the potential to make lectures interesting. Indeed, I am determined to use teaching aids more often in my lectures. However, I also think that the words “appropriate use” need to be emphasised bit more strongly within this context. In fact, in their seminal book, Brown and Atkins (1988) discuss specifically about tactics to make the lectures in higher education “interesting”. Not surprisingly, in their book, one of the most cited tactics is to use teaching aids. However, they specifically advise to refrain from using highly sophisticated teaching aids in the early stages of the lecture, as that would lead to information overloading resulting in quite the opposite effect to that one would expect.
I guess this brings us to the million pound question, to prop or not to prop? There won’t be one right or wrong answer I think, it depends on the circumstances. However, evaluating various options in teaching and understanding how students learn are important aspects in becoming a good teacher (UKPSF K3 – how students learn, both generally and within their subject /disciplinary area(s)). In my case, I will use teaching aids in my lectures much more than ever before, but not before thinking twice about the suitability of the aid to the circumstance and especially the audience.
[image from: http://www.plink.co.uk/wordpress/wp-content/uploads/2010/10/Picture-13.png]
References
- Brown, G. and M. Atkins (1988).Effective teaching in higher education, Routledge.
- Reid, D. and M. Johnston (1999). “Improving teaching in higher education: Student and teacher perspectives.”Educational Studies (3): 269-281.
Lets’ complicate the things a bit more: creativity in international joint curriculum development
EURASIA joint curriculum development approach
Prelude
Week 3 theme for the PGCAP core module was on lessons planning and curriculum design. While the in-class discussions were more leaned towards “planning a session”, the reading tasks for the week were focused mainly on “creativity in curriculum design” (Oliver 2002). This is a subject that I have been thinking and researching about during last few years. As identified in UKPSK framework, it is important to understand how research can inform teaching practices (UKPSF A5 – “engage in continuing professional development in subjects/disciplines and their pedagogy, incorporating research, scholarship and the evaluation of professional practices” and UKPSF V3 – “use evidence-informed approaches and the outcomes from research, scholarship and continuing professional development”)
Hence, I will base this week’s blog reflecting upon that a bit.
Background
Indeed, at the very centre of the concept of “creativity”, the fuzzy nature of its definition is etched. Oliver (2002)’s interview findings highlight these variety of view-points about “creativity” within various curriculum design approaches. Guided by the social constructionism view (Jonassen 1991), I have attempted to map the concept of “creativity” in curriculum design to my own experience. In that process, I particularly found the concept of “creativity as value-driven breaks with tradition” (Oliver 2002) closer to my view of creativity within the scope of curriculum design.
My exposure to curriculum design mainly comes from two European Union funded research projects. The first project is EURASIA (www.eurasia.salford.ac.uk), an EU funded collaborative research project completed in 2008 (at the time, I worked as the researcher for this project from the Salford end). The second project is BELLCURVE (www.disaster-resilience.salford.ac.uk/bellcurve), another EU funded collaborative research project, which is currently on-going (I am one of the co-investigators for this project). To manage the length of this article, I will focus on my EURASIA experience within this post (perhaps another post on BELLCURVE later).
EURAISA and joint curriculum: where is the creativity?
The EURopean and ASian Infrastructure Advantage (EURASIA) is an international collaborative research project with the specific aim of addressing the above highlighted requirement. Five project partners are working in collaboration within this project; three European higher education institutes and two Sri Lankan higher education institutes. EURASIA project aimed to enhance the capacity of the partner institutions for training, teaching and research activities required for the creation and long-term management of public and commercial facilities and infrastructure. The overall objective of the project is to foster cooperation in Higher Education institutions in both Europe and Asia, improve reciprocal understanding of cultures, exchange best practice and strengthen mutual awareness of programmes.
One of the main tasks (one of 7 work packages) of the EURAISA project was to develop joint curriculum on facilities and infrastructure management, to be incorporated to various programmes at all partner institutions. Was curriculum design task “creative”? It was expected to be a “joint” curriculum across the five different higher education institutions; hence there was a clear difference between this design process and the “traditional” approach.
Going by the “creativity as value-driven breaks with tradition” (Oliver 2002), one can argue that this curriculum design /develop was, indeed “creative”). However, the requirement here is to test whether this “break” from the tradition would actually be able to be classified as a “value-driven” break.
The main justification for a joint curriculum proposed within the EURASIA project is related to good practice and knowledge sharing across counties (See: Keraminiyage et al. 2008). While the “value” created through this joint curriculum may not directly contributed to the learning experience of the student (with regards to “how” they learn), it has undoubtedly had an impact on the quality of information communicated through the module(s) delivery. In that sense, the scope of the term “value” defined within Oliver (2002)’s article may need to extend to include not only “how” but also “what” the students learn through the curriculum (as it stands now, there seems to be a strong link implied between the term “value” with “how” the students learn, but less emphasis on what they learn when defining “creativity” .
Due to the collaborative nature of the project and the expected characteristics of the joint curriculum, the design process was also different from the “traditional” approach (perhaps this can be loosely attributed to Oliver (2002)’s concept of creativity as ‘doing creation’). The EURASIA approach to joint curriculum design is illustrated below.
Within this research, there are two specific characteristics identified with reference to joint curriculum development in general:
- There is a tendency that the final outcome would be a generic and imbalanced curriculum owing to the fact that, it has been designed to cater for different objectives and needs of different participants.
- The curriculum needs to be flexible enough to cater for different market, quality and skills requirements under different contexts.
Guided by the characteristics identified, as evident from the illustration above, EURASIA development process consist of two main phases. The first phase (downward arrows) was to collect information and design / develop the common curriculum and the second phase (upward arrows) was to customise the curriculum to suit individual partner needs and skill sets. The principle behind this was to acknowledge the fact that, good curricular are indeed, context specific, yet, lessons can be learnt from other contexts to promote creativity within our own.
Obstacles and observations
Experience gained through EURASIA further reinforced Oliver (2002)’s observations above the issue of “inherited” curriculum and sense of ownership in curriculum design. Having based on a large pool of modules / curricular data (see BOX 3 from the top in figure 1), the EURASIA curriculum development process provided a rich case environment to test the issue of inherited modules in curriculum development. The main difficulty was on deciding (jointly) the base and the structure of the common joint curriculum, by evaluating a large number of individualistic (different partners) module specifications and requirements. Similar feelings expressed by the interviewees in Oliver (2002)’s study were present, when evaluating the partners’ comments and views about the developed common joint curriculum. These comments often referred to the concerns about tutors’ ability (as well as the willingness and the encouragement) to deliver a curriculum developed outside their individual contexts and without their direct input. Within EURASIA these issues were resolved to a larger extent by partners the flexibility to customise the common curriculum to suit their individual requirements and skill sets. The module evaluation mechanisms and parameters are further discussed in Keraminiyage et al. (2008).
Re-living the moments of joint module development within the EURAISA project, combined with the new insights developed through the discussions and readings under the PGCAP, I now started thinking “could we have done more” to make the EURASIA joint curriculum more “creative”? I guess, we never stop learning, yet we have to live the present. Your comments are appreciated.
References
Jonassen, D. H. (1991). “Objectivism versus constructivism: Do we need a new philosophical paradigm?” Educational technology research and development 39(3): 5-14.
Keraminiyage, K., R. Amaratunga, et al. (2008). Enhancing the capacity of Sri Lankan higher education institutions to meet post disaster recovery challenges. iREC. Christchurch, New Zealand.
Oliver, M. (2002). “Creativity and the curriculum design process: a case study.” Retrieved October 14, 2011, from www.heacademy.ac.uk/resources.asp.
Kaushal's PGCAP e-portfolio 
You must be logged in to post a comment.