Open and distance e-Learning (ODeL) in higher education has been transforming our traditional view of learning not just in the delivery of courses but also in the method of teaching, learning, and assessment. Just like in the traditional setting, peer assessment is also an integral part of online learning. Peer assessment in the learning process must enhance interpretation and reflection, develop critical thinking skills, and accept peer criticism. Aside from providing better understanding of students’ needs, development, and achievement, effective and reliable peer assessment can also be a tool for co-creation of academic text. As such, peer assessment is also a means for knowledge sharing and creation among teachers and learners. Thus, peer assessment should be perceived as ‘assessment for learning’ more than ‘assessment of learning’.
According to Brooke and Andrade (2013), peer assessment is defined by the following:
“simply a matter of students giving informed feedback to one another on an assignment. Effective peer assessment is related to clear standards and is supported by a constructive process of critique. Peer assessment is a valuable tool because feedback from peers can be delivered with more immediacy and in greater volume than teacher feedback. Peer assessment should happen during the learning process, on works-in-progress, and be followed by opportunities for students to use the feedback they received to revise their work” (Brooke & Andrade, 2013; p. 1)
Thus, peer assessment should not be a process by which peers determine grades for one another.
Peer assessment enhances student interpretation and reflection. Students are able to observe their peers’ work and learning process, and therefore come to a better understanding of how their peers learn. In the process of peer assessment, students are capable of learning how to make judgment, criticize peer work, and accept peer criticism, thereby developing their critical thinking skills (Lan et al., 2007; Kwok & Ma, 1999).
Brooke and Andrade (2013; p. 2) also listed the guidelines for effective peer assessment as follows:
1. Determine the criteria on which the assignment will be assessed (what counts).
This can be done by the teacher alone, or preferably by co-constructing a checklist or rubric with students.
2. The teacher groups students into small peer feedback groups.
Two to four students can be grouped based on ability level.
3. The teacher may take an assignment and use the Ladder of Feedback protocol to ask clarifying questions, state what she or he values about the assignment, list what concerns her or him about it, and ultimately make suggestions (not mandates) that may be used to improve the assignment.
4. Students receive a checklist or document that reminds them how to deliver effective peer feedback. The Ladder of Feedback is a good choice of a checklist for students to use as a quick reference.
5. The teacher clarifies the assignment to the students. Clarification includes peer assessment of performance and the timeline for that assessment.
6. The teacher actively monitors the progress of the peer feedback groups.
Students will need a lot of support when they are first introduced to peer assessment, and less as they become accustomed to it.
7. The teacher monitors the quality of feedback.
The teacher ensures that her students are using the constructive feedback protocol (possibly the Ladder of Feedback).
8. Peer feedback is checked for reliability.
The teacher may compare she or he feedback on an assignment with a student’s feedback to check for alignment and provide further support and instruction if needed.
9. The teacher provides feedback to students on the effectiveness of their peer assessment.
10. After students have generated effective peer feedback, it should be used to guide student revisions of works-in-progress.
Similarly, Yeh and colleagues (2008), state that “peer assessment becomes a popular and effective approach to enhance students’ critical thinking skills and make them reflect more about their and others’ work.” The study they conducted showed how web-based peer assessment systems were developed and applied to assist in assessing peers’ works in several school or university activities. The application of peer assessment encouraged the students to deeply reflect, evaluate, and comment on peers’ work. It was also noted, based on some studies, that peer feedback will considerably “strengthen students’ academic writing skills as well as foster meta-cognitive awareness on learning processes” (Gibbs et al. 2004, Nicol et. al 2006, Jensen, 2017).
Open and distance e-Learning (ODeL) is a worldview of teaching and learning that is grounded on three (3) main aspects: the access, equity, and resource sharing philosophy of open learning; the learner-centered and flexible technology mediation in distance education; and the connectivity, ubiquity, and interactivity approaches of e-Learning, while being supported by the values that underpin the university ethos – academic excellence, academic freedom, intellectual pluralism, and service to society (Alfonso, 2014).
Just like in a traditional education setting, peer assessment is also an integral part of an online learning environment. A study that looked at the benefits of the group or self-evaluation integrated in an open and distance learning system particularly states that group discussion is a vital aspect of study skills in an open and distance learning environment which involves self, peer, and group participation (Usuji, U.S.A.,2012).
Peer or group self-assessment of assignments, projects and/or tests can improve the learning experiences of the students (Usuji, U.S.A., 2012 cited in Boud 1995; Brown & Knight, 1994; Gibbs, 1992; Brown, Bull & Pendlebury, 1997; Brown & Glasner, 1999; Brown & Dove, 1990). Again, this can be utilized in helping the students develop critical thinking during the evaluation process.
Though peer-assessment is recognized as one of the useful tools for evaluating learning and skills development in an open and distance e-Learning setting, there are also challenges in this kind of assessment. For one, learners may not have enough knowledge and experiences to evaluate their peer’s work. Another concern is the reliability or quality of learner’s feedback. It is important that the learners accept accountability and provide careful comments; however, not everyone gets involved (Hadaddi, et al., 2017).
In this course of the UP Open University, peer assessment was introduced in several online courses under the Bachelor of Arts in Multimedia Studies (BAMS) program. One of the major courses under BAMS program that employs peer assessment as a tool for co-creation of academic texts is Videography in Multimedia (MMS 175). In this course there is an emphasis on the teacher and learner as multimedia researchers and co-creators of academic text. MMS 175 focuses on the process and techniques of producing videos with focus on the visual language. Principles of videography are discussed extensively to provide a framework to ground the capturing of images in the student’s video outputs. The theorizing on videography as form of research, documentary, and narrative-telling brings students in to higher level of innovation, creativity, and critical thinking fit to critique and evaluate their works and the works of others.
Furthermore, the course focuses on capacitating students in knowing what images to capture. The importance of the concept, “know how to see”, and automatically use these principles of visual design as part of the process of capturing the moving images through video footage is emphasized. Course outputs of the students are largely concentrated on principles of design that highlight the importance of taking up the aspects such as frame, shot, sequence, scene, time, pace, rhythm, mise-en-scène, and montage among others which are present in all video works.
The course navigates the students through the process of generating videos as part of multimedia productions. As an exercise, they are asked to produce a short video using montage of short takes, and a short video making use of long take. This emphasizes that the doing of videography highlights its praxis. The way multimedia productions capture process, emotion, concept, and story under the different genres are showcased in this course. There are six (6) genres employed in this course: learning objects, informational materials, documentaries, animation, experimental, and narrative. This course addresses the development of skills and awareness of theorizing in multimedia production. Going through this exercise-driven course design brings the students in doing their exercises through open fora, synthesis papers, reflection papers and short video exercises to finally prepare them for the peer assessed final video production.
As part of the course requirements, the students are required to produce their own video output. A very brief version of the annotation or write-up of their work and the URL of final video production in the forum in myPortal are shared with their fellow learners. Viewing is scheduled, and the same forum is used to give insights on their classmates’ productions. Furthermore, the forum is a venue for sharing and inviting their classmates to watch each other’s work.
After watching the video productions of their classmates, students are required to submit in the assignment bin their “Top Ten” choices, with reasons for each choice. This is considered the final assignment in the course.
Students’ Insights
The top ten choices of the students are listed and ranked accordingly. Out of the list, the top five video productions are selected. In this particular instance, insights of the students on the top two videos were obtained.
Top 1 video:
“The narration in this video is what makes it unique against the other video productions in this list. The cuts were done in tune with the narration and the whole production had a rhythm to it that gave it a good sense of flow. The angles taken were also very interesting as well as the blurry and moving shots. Overall this is an excellent production but I think a more muted approach in color editing would make the visuals more in tune with the audio.”
“The short film was relatable; he was able to apply all the good elements to create a good video – depth of field, panning, zoom, colors, looking space, extreme close up and lastly his storytelling and/or narration fits the video.”
“Works that tugs the audience’s heartstrings like this is something I’ve always admired. This is an excellent video, good audio quality; elements of design can be noted.”
“The video had an indie feel to it. It was raw but it looked professional. I can see that he applied the different camera shots and camera movements that were taught to us.”
“I love how he has used the right scenes to emphasize the message that he is talking about. Coupled with great videography, I believe he was effective in bringing out emotions from the video itself and channel it to the feelings of the people who are watching it. I was moved by the poetic prowess of the video creator and the message that is being sent out by the video.”
“A video that is mainly aimed at showing a poet’s deep inner conflict is an interesting and melancholic piece for the entertainment and engagement of the viewers. Subtitles are perfectly matched to the clips that bring visual and vivid artistic interpretation to the lines of the poem. The use of pitch black screen times is also clever for this can create a subtle but felt feeling and experience. The narrator’s voice is great, his is handsome and probably many girls will fall in love with the voice.”
“Long shot, medium shot and close up shots are some of the shots being applied in his film.
- In terms of camera movements, there are some follow shot having a bit small of unsteadiness in his camera, although the application of smoothness is still there.
- He uses more on panning and tilting
- In terms of Camera Placement
- Application of Low and High angle shots
- In terms of Composition
- He applied the rule of thirds
- And the depth of field composition”
Top 2 video:
“The plot is simple but it is incredible in a way that many of its viewers can relate to it. The story of expressing our feelings will always be a timeless experience for us human beings. Stop motion animation is indeed a challenging genre. From what I saw from this clip, there were numerous picture frames that were employed to accomplish this project. Only a truly dedicated video creator can do this for the job calls for an uncanny eye for detail. The use of Pokémon music is also a good idea as this production can appeal to those who still have their hearts as a child.”
“The video was playfully silly and feel good at most. I am not discounting the effort of the creator but I am just saying that I felt good all the time that I was watching the project. Kudos for putting the effort in this awesome project and the typical romantic setup made it look cheesier but refreshing to watch.”
“I like the video because it seems like I am watching a live game in a different way. I love how creative she is organizing all her collections so she can show it in her video.”
“I picked this video because of the effects and camera shots that were incorporated in it. There were minimal movements from the inanimate objects themselves which did not make the movements look smooth but the changes in focus and angels were clean.”
“Good background music, good angles used, and much effort was given to complete this wonderful video. I am giggling like a kid while watching this video. I love and enjoyed it so much!”
“I have always been fascinated with stop motion animation video, and I admire the people behind it. Paola’s shot were clear; depth of field is on point also her close up shots. Storytelling and post video editing was cool. The over the shoulder shots, and the depth of field were applied.”
Teacher’s Insight
Upon reading and evaluating the insights and comments of the students on their classmate’s video outputs, the following realizations were concluded in the application of peer assessment:
- Peer assessment promotes different perspectives in learning
Peer assessment also called peer grading, peer review, or peer evaluation is commonly used in teaching whether it be a face-to-face or online mode of education. It is a process of evaluating another person’s work that does not only involve one’s own opinion but also critical thinking based on certain criteria and rubrics. It is beyond mere looking—it is seeing with careful attention to and respect for the work. We think that it is mostly from the teacher that the learning emanates; but with peer assessment, we see students as source of knowledge and learning too. Students are able to assess others’ and their own work the same time realizing things they did not know before, thereby gaining new learnings from their classmates. Students are able to impart knowledge to each other.
- Peer assessment validates teacher’s assessment
The learning of the student can be efficiently evaluated and presented through different forms of assessment like peer assessment. In a traditional setting, it is the pen-and-paper form of assessments that the teacher mostly relies on. Results of the peer assessment may imply several things such as which part of the learning process needs improvement and which practices are effective.
- Peer assessment reflects student’s strengths and weaknesses
The process of students evaluating one another reflects their own learning of the concepts based on how the learner uses it in commenting on others’ work. On the part of the teacher, it gives an idea on how the students looked at and takes effort to give comments and suggestions to improve peer’s work. There are students who apply critical thinking skills to focus not just on the technical aspects but also give emphasis or importance on the message or content and the quality of the work as a whole. It also identifies students who know how to pay attention to details and those who do not. It also shows the learners who might need help or support in improving either the theoretical or the technical skills.
- Peer assessment enhances the process of learning
Careful integration of peer assessment in a course can enhance students’ learning in numerous ways. It gains and builds support community; it promotes thoughtful and reflective discussions among learners; and most importantly, it fosters more capacity for higher critical cognitive skills and valuable judgement. Through eliciting suggestions for further improvement, students are able to boost each other’s confidence using constructive criticism.
Peer assessment is part of the direction of more openness in assessment design. This is going towards openness where there is learner participation and use of measurement parameters created by a homogenous population which in this case is a class on videography. This is consonant with our times of creating digital teaching and learning communities. This chapter focuses on a process of evaluation and assessment that is grounded in an ODeL environment. This gives space to learners as co-creators of academic text together with their faculty to make use of a teaching and learning process considered as part of open pedagogy.
This process of assessment brings us closer to measuring knowledge creation, innovation, and creativity through this open teaching and learning course design that is activity-based. Learners who have been closely honed throughout the course become familiar with videography through practice and praxis. The learners, as they become videographers, are likewise turned into reviewers, critics, and evaluators allowed by this course design that maximizes the connectivity, interactivity, and ubiquity of digital technology, and digital pedagogy as well.
Open and Distance Learning (ODL) allows this openness maximizing the affordances of e-Learning as experienced in ODeL as platform and space for multimedia as research and production of content. However, it is even more than that, since the works produced are both art and science as well. Multimedia enhances the teaching and learning experiences of both open universities and traditional brick-and-mortar higher education institutions as they integrate ODeL and peer evaluation as one of the forms of assessment. Multimedia merges oral, visual, and written texts, and dictates new methodologies and modalities. This is a call for a continuing study of peer evaluation as a form of assessment in this ever changing digital environment and it is an important part of a multi-modal evaluation and assessment design.
Admiraal, W, Huisman, B and Pilli, O. (2015). Assessment in Massive Open Online Courses The Electronic Journal of e-Learning. 13(2), pp207-216. Retrieved from http://ejel.org/volume13/issue4/p207
Alfonso, Grace J. (2014). Creating spaces and possibilities through open and distance elearning (ODeL): A worldview. In Alfonso, G.J. & Garcia, P.G. (Eds.). ODeL Open and Distance eLearning: Shaping the Future of Teaching and Learning. BAMS program proposal presented to the UPOU University Council meeting dated 15 August 2017.
Brooke, E.G. & Andrade, H. (2013). Student-centered assessment guide: Peer assessment. Boston, MA: Jobs for the Future. Retrieved from https://studentsatthecenterhub.org/resource/student-centered-assessment-guide-peer-assessment/
Gouli, E., Gogoulou, A., & Grigoriadou, M. (2008). Supporting self-, peer-, and collaborative-assessment in e-learning: The case of the Peer and collaborative assessment environment (PECASSE). Journal of Interactive Learning Research, 19(4), 615-647. Retrieved from https://search.proquest.com/docview/211313852?accountid=47253
Haddadi, L., Bouarab-Dahmani, F., Berkane, T., & Lazib, S. (2017). A new method for peer assessment in MOOCS. Paper presented at the International Scientific Conference, 1416-423. doi:http://dx.doi.org/10.12753/2066-026X-17-061
Heinrich, E. (2004). Electronic repositories of marked student work and their contributions to formative evaluation. Journal of Educational Technology & Society, 7(3), 82-96. Retrieved from https://search.proquest.com/docview/1287054246?accountid=47253
Jensen, T. W., & Dankl, K. (2017). Peer feedback: Using disciplinary-specific teaching formats as “bridges” to enable student engagement. DNU Conference 2017. Retrieved from http://dun-net.dk/media/304874/pa-peer-feedback-using-diciplinary-specific-teaching-formats-as-bridges-to-enable-student-engagement.pdf
Jimenez-Romero, C., Johnson, J., & De Castro, R. (2013). Machine and social intelligent peer-assessment systems for assessing large student populations in massive open online education. Paper presented at the 598-XIV. Retrieved from https://search.proquest.com/docview/1467835949?accountid=47253
Lacatus, M. L., Staiculescu, C., & Cace, C. (2017). Pedagogy issues in MOOCS: challenges in assessment in online economics courses.Paper presented at the International Scientific Conference, 283-288. doi:http://dx.doi.org/10.12753/2066-026X-17-125
Murray, J., & Boyd, S. (2015). A Preliminary Evaluation of Using WebPA for Online Peer Assessment of Collaborative Performance by Groups of Online Distance Learners. International Journal Of E-Learning & Distance Education, 30(2), page range and DOI.
Olofsson, A. D., Lindberg, J. O., & Trond, E. H. (2011). Blogs and the design of reflective peer-to-peer technology-enhanced learning and formative assessment. Campus – Wide Information Systems, 28(3), 183-194. doi:http://dx.doi.org/10.1108/10650741111145715
Yeh, Shiau-Ping & Liu, Tzu-Chien & Graf, Sabine & Wang, Yu. (2008). Exploring the Development of Web-based Peer Assessment System. Retrieved from: https://www.researchgate.net/publication/252490810_Exploring_the_Development_of_Web-based_Peer_Assessment_System
SUEN, Hoi K..Peer assessment for massive open online courses (MOOCs). The International Review of Research in Open and Distributed Learning, 15(3). ISSN 1492-3831. Retrieved from: <http://www.irrodl.org/index.php/irrodl/article/view/1680/2904>. Date accessed: 05 Oct. 2017. doi:http://dx.doi.org/10.19173/irrodl.v15i3.1680.
Alfonso, G. J., Suarez. M. J., & Bundalian, C. B. (2018). Peer Assessment as a Tool for Co-creation of Academic Text in ODEL (Eds.), Assessment Praxis in Open and Distance e-Learning: Thoughts and Practices in UPOU (pp. 87-98). Los Baños, Laguna, Philippines: UP Open University