In this chapter we argue that in Open and Distance e-Learning (ODeL), assessment is at the core and of primary consideration. Assessment is taken as a valuation process that constructs not just the learner’s but also the educational institution’s identity. The principles of quality and credibility of assessment in an ODeL environment are explored in developing an assessment system at the course and at the program levels of the ASEAN Studies Graduate Program, the framework of which was collaboratively developed by five Open Universities in the ASEAN region. At the course level, the learning outcome, which is a research proposal, shall be assessed through the use of three markers to establish reliability of the valuation process; at the program level, an essay test shall be used to assess completion of three program objectives, and it shall be evaluated by a panel coming from the four other Open Universities. The chapter also addresses two issues related to ODeL: authentic learner identification and scalability of the conceptualized assessment system.
Open and distance e-Learning (ODeL) draws from the philosophy of openness, inclusion, resource sharing, access and equity of open learning and learner-centeredness, flexibility, active learning, interactivity, ubiquity, connectivity, and constructivist view of e-learning. These affordances and features are infused with values that underpin the universitas – excellence, academic freedom, humanism, intellectual pluralism, democracy, and service to society. These ethos create the spirit of the university.
Together, all these elements are embedded and facilitated by networked information and communication technologies and make up what is referred to as ODeL. The interweaving of these components can bring about social transformation by producing learners who have the ability to seek new information from different sources, translate this information into applicable knowledge, and communicate this knowledge in various forms and situations. They should be able to work in different cultural contexts as well. ODeL with its affordances can help address the challenges of education in the 21st century, more particularly ubiquitous learning. The emergence of e-mobile culture has enhanced the ubiquitous nature of Information and Communication Technology (ICT) based on the ODeL framework.
Today, industries and communities of practice are turning to the higher education sector for massive, yet more specific and tailor-made courses and programs in their fields. These learners prefer to study part-time and acquire the knowledge necessary to advance themselves in their careers. Likewise, there is a need to recognize the changing learning styles of students. Each year, more and more digital natives are joining the higher education sector. Even digital migrants are constantly being put in a position where they have to adapt to a new technology.
Despite the preponderance of educational resources on the internet and the growing number of privately run e-learning providers, universities in general, and open universities in particular, will always play a central role in the accreditation of student learning particularly for transnational education. Universities can only continue to perform this accreditation role if their ways of assessing their students’ learning remain appropriate and credible in the eyes of our stakeholders.
The growth of the internet users and mobile applications in the virtual environment has created the need for authentic assessment addressing the realities brought about by a web-based, technology-driven, and potentially massive teaching and learning situation. To a certain extent, the stage is set for the ODeL framework to work and be maximized.
Assessment of learning outcome is a valuation process where the learner’s value is judged by the educator. Assessment, as we know, is a process of collecting information and making a judgment, identifying the strengths and weaknesses, the good and the bad, and the right and the wrong, in some cases. It is more than simply giving marks or grades (Rust, 2002); rather, assessment must address all learning objectives, be fair to every learner, and must be transparent to those who will be assessed. Moreover, the quality of measuring instruments to collect the information must be established through reliability and validity measures. Validity is the extent to which a test or instrument measures what it is supposed to measure and reliability is the degree to which an assessment tool consistently measures whatever it measures (Gay, 1987). The more educators provide good assessment, the more students receive good judgment.
While the principles involved in assessment of learning remain the same, there are unique challenges in assessment for ODeL. In addition to the given assessment features, the nature of and potential massive reach of online learning has to be taken into consideration, thus requiring appropriate administration processes. It is recognized that while technology provides us with a huge advantage in reaching our intended learners, it also has vulnerabilities associated with establishing the identity of the person undergoing assessment.
Online exam is a process that allows learners to take the examination from a distance. With today’s technology, examinations could be administered through the internet. In the case of Sukhothai Thammatirat Open University (STOU) in Thailand, computer rooms for e-exam are set up in remote campuses where learners take their exams in testing centers nearest them. A system has been put in place where transmission of test answers to the main campus is done instantaneously through the internet (Jeotee, 2014). In Cambridge Towards an Assessment System for Open and Distance e-Learning: The Case of the ASEAN Studies Graduate Program University, the exam is uploaded to ‘Cloud system’ and each station downloads the exam for their examinees. When the exams have been done, they will immediately be transmitted to Cambridge University (Lebus, 2011). Meanwhile, in the UP Open University, the learners are allowed to take online exams at designated Learning/ Testing Centers, Philippine embassies for those who are abroad, and in special cases, in their own homes. Monitoring from UP Open University campus is done through a computer camera installed by the exam taker and approved by the UP Open University proctor.
On one hand, these examples show the advantage of technology; on the other, the online nature of learning environment predisposes the assessment system to some weaknesses such as: learners can easily violate exam rules, save the test for other purposes, allow someone to help them answer the test through the internet system, or cheat in some way. Even as efforts are exerted to ensure credibility of the assessment procedure as seen in the examples cited, there remain serious challenges related to the authentication of the learner’s identity in the context of ODeL.
To maximize the affordances provided by the web, the UP Open University started to adopt online student support in 2001, went fully online in 2007 turning its courses into resource-based packages, and made online teaching as the main approach to instruction. UP Open University grew together with the internet phenomenon allowing access all over the world of online digitized academic texts.
Given these considerations and realities, this chapter considers an online assessment system for a program that requires an innovative approach to capture the nuances of ODeL applied to a new Master of Asean Studies graduate program (MAS) collaboratively developed by a consortium of five open universities in the ASEAN region and offered beginning August 2014. The five universities collaborating in this program are: Universitas Terbuka (Indonesia), Open University of Malaysia, the UP Open University (Philippines), Sukhothai Thammathirat Open University (Thailand), and the Hanoi Open University (Vietnam).
The Master of ASEAN Studies (MAS) aims to promote regional and global understanding about ASEAN countries from the perspective of ASEAN. With the ASEAN integration in 2015, it is expected to attract students from diverse groups worldwide making ODeL relevant. The unique terms of the collaborative nature of the development and delivery of the program suggest that assessment should consider quality and credibility of online assessment and scalability issues.
The Master of ASEAN Studies program consists of 27 units of coursework, 6 units of thesis, and 1 unit of colloquium. Given the program structure, assessment can be based on module-level, unit-level, course-level, or program-level objectives. As previously discussed, any respectable assessment scheme is informed by the learning objectives, activities, and outcomes articulated in the course syllabus and in the program description. For purposes of this chapter, we focus on course-level objectives as well as on program-level objectives to illustrate the important and unique elements of ODeL assessment.
Being a graduate program, MAS requires a research course. Its uniqueness is that it is a course that will have as its final output an Aseanology research proposal. It requires an understanding of the concept of Aseanology and its implication for and application to research. The research course has five course-level objectives that span the levels of expected competencies involving (1) understanding, (2) application, (3) analysis, (4) critical evaluation, and (5) creation. While the objectives are laid out in the order of least to highest competency levels, the activities required in the course all lead up to the development of an Aseanology research proposal.
Since the research proposal is the ultimate course requirement, it will be the focus of the summative assessment of the course. Learning activities related to the first four objectives need not be graded but assessment will be in the form of giving feedback towards achievement of course objective number 5. It is therefore argued that for a given course, assessment could have a duality of purpose — assessment in the form of feedback and assessment in the form of valuation.
A valuation procedure will include contributions from three sources: faculty-in-charge (FIC); tutor; and fellow learner under a blind review process. Students will be randomly assigned a fellow student’s research proposal for evaluation using a rubric with the additional requirement of justifying the score given. In this manner, students are given another opportunity in the learning process to show what they have learned. Reliability measures could then be derived from these sources to ensure fair and transparent assessment.
At the program level, the student is required to take a comprehensive examination after taking 27 units of courses. The comprehensive examination will assess accomplishment of three of the five MAS program objectives:
- to demonstrate comprehensive understanding, insights, and mastery of ASEAN and the countries comprising it;
- to contribute to an enhanced and heightened ASEAN community’s consciousness and awareness of its ties of history, culture and bound by a common regional identity but celebrating diversity; and
- to apply balanced theoretical and practical perspectives in engaging with issues and problems in the ASEAN region.
An essay test is considered an appropriate assessment tool for the comprehensive exam as the process of writing an essay involves analytical, critical thinking, and communication skills which are suggestive of a deep learning approach. It allows students control over the selection, organization and presentation of their knowledge and understanding; and challenges them “to present conditional and metatheoretical knowledge if they possess it” (Boulton-Lewis 1995: 146 as cited by Scouler, 1998).
Essay exams can thus provide evidence of understanding and acquisition of ASEAN consciousness and perspective and thus, meet the standard of validity of assessment. A panel from the other four collaborating open universities involved in ASEAN Studies shall evaluate answers to the essay examination to further ensure quality of assessment and in observance of the principles of fairness and transparency.
Besides the quality of assessment, the credibility of online assessment and scalability are issues that an ODeL program should address.
A possible step is to require students to register ten fingerprints when they apply to the university as a means to authenticate the identity of the learner undergoing assessment. A proctor shall ask the student to scan any one or two of the ten fingers that will be compared to the fingerprint biography database at the university. Student data such as picture, address, and nationality shall be available for the proctor in processing fingerprint approval. Students who will undergo assessment shall be required to use a camera that will record the whole exam session. Such recording will be consulted in times when there is doubt about the assessment result. In cases where the assessment procedure is of objective nature, a computer program, such as Safe Exam Browser which limits copying, surfing, and saving capacity of the learner’s computer, shall be used.
In the context of ODeL, students should be able to undergo assessment anytime and anywhere. Thus, scalability (being able to provide assessment to a big number of students at a time) is another vital issue that needs to be addressed. A means for the MAS program to ensure a fair scalable assessment is to develop an assessment bank where numerous assessment sets can be easily drawn that have similar characteristics, such as same assessment objectives, same number of items, same criteria and same grading scheme.
In this chapter, we argue that in open and distance e-Learning (ODeL), assessment is at the core and of primary consideration. Assessment is taken as a valuation process that constructs not just the learner’s but also the educational institution’s identity. Thus, preparing quality and credible online assessment requires educators to carefully observe the qualities of validity and of reliability and the principles of fairness and of transparency, as well as an awareness of the security and of the scalability of the conceptualized assessment system.
Boulton-Lewis, G.M. (1995). The SOLO taxonomy as a means of shaping and assessing learning in higher education. Higher Education Research and Development 14(2), 143–154.
Gay, L. (1987). Educational research: competencies for analysis and application. Merrill Pub. Co., Columbus. Jeotee, K. (2014). Trends in distance examinations in an IT age. Inaugural International Conference on Open and Flexible Education (ICOFE 2014) Proceeding. The Open University of Hong Kong, Hong Kong SAR, China.
Lebus, S. (2011). E-assessment for 21st Century Learning: E-assessment in high stakes examinations. Retrieved video clip from: https://www.youtube.com/watch?v=sb6mFK D19V8.
Rust, C. (2002). Purposes and principles of assessment. The Oxford Centre for Staff and Learning Development, Oxford Brookes University. Wheatley Campus Oxford OX33 1HX
Scouller, K. (1998). The influence of assessment method on students’ learning approaches: Multiple choice question examination versus assignment essay. Higher Education, 35(4)
Saludadez, J. A., Jeotee, K., Alfonso, G. J., & Lumanda, M. F. (2018). Towards an Assessment System for Open and Distance e-Learning: The Case of the ASEAN Studies Graduate Program. In M. F. Lumanta, & L. C. Carascal (Eds.), Assessment Praxis in Open and Distance e-Learning: Thoughts and Practices in UPOU (pp. 45-52). Los Baños, Laguna, Philippines: UP Open University