In this culminating chapter, it is suggested that quality considerations in every aspect of an assessment system in an open and distance e-Learning (ODeL) institution be incorporated in a multi-level, multi-function assessment framework. Levels of assessment are often discussed as assessment at course level, assessment at programme level, and assessment at institution level. Assessment at each level serves various and distinct purposes often referred to as, but not limited to, assessment of learning, assessment as learning, and assessment for learning. 

Quality assessment is a process, engaged in by an institution, of evaluating its activities with the end in view of continuous improvement. In higher education, assessment, as concept and practice, may be viewed from a multi-level and multi-functional perspective to include assessment of learners within the course, assessment of programme and its alignment with mission of the institution, and assessment at the institution level as it addresses the needs of its various stakeholders. In this chapter, the authors attempt to identify dimensions that may lead to the development of a quality assessment system in the context of ODeL.

Assessment in its various forms has traditionally been discussed in the context of classroom-based education. More recently, however, with the introduction of open and distance e-Learning (ODeL), greater focus has been directed at quality considerations of an assessment system brought about by the unique circumstances of openness, distance, and technological advances. Variously referred to as distance education, open learning, distributed learning, flexible learning, and e-Learning (Alfonso & Garcia, 2015; Keegan, 1980), this teaching and learning approach capitalizes on independence of learners and increased flexibility in teaching methodologies necessitating greater openness in assessment at various levels. While in the past, quality assessment was based on a set of “fixed criteria” applied to the entire educational system, there is greater recognition now for a more adaptable set of “flexible criteria” (Reisberg, 2010).

In ODeL, the issues related to quality assessment are usually addressed in terms of technological innovations that impact on the teaching and learning process. The affordances of open learning, distance education and e-learning combined in the ODeL framework (Figure 1) include: “access and equity, resource sharing, learner-centeredness, flexibility, active learning, interactivity, ubiquity, and connectivity” (Alfonso, 2015). It becomes imperative, therefore, that in a quality assessment framework these are taken into consideration.

Levels of assessment often refer to the course, programme, and institutional levels. Miller and Leskes (2005) conceptualized levels of assessment from student to institution resulting in five levels of assessment. The first three of these refer to student and course level assessment where summative and formative assessment are employed. The authors discussed assessing student learning within courses, assessing student learning across courses, and assessing courses.

Evidence–based practice in teaching and learning engages various assessment information to monitor and/or assess student achievement and effectiveness of teaching, instructional programmes, or even the entire institution. These assessments provide guidance for possible adjustments in future instructional decisions, can be categorized into multiple types and levels, and can be considered in the University’s comprehensive assessment system. Currently, data-driven decision making as an ongoing process of analyzing and evaluating data informs educational decisions related to planning and instructional strategies at various levels. However, institutions would also use information that are not directly linked to student learning (e.g., teaching load, facilities, external collaboration) and include them in a comprehensive analysis of the academic environment, particularly at the programme and institutional levels.

In assessing student learning within courses, formative assessment is utilized in determining the students’ individual learning while summative assessment is used to identify how well they are meeting the goals of the course (Rahman & Ramos, 2003). Data gathered are usually sequential, since evidence is collected during and at the end of the course to monitor progress, thereby summing up a semester of learning. Evidence of student learning can be used for feedback to the course instructor’s ability to communicate with and motivate her or his students. The instructors’ duty also extends to assessing the quality of their teaching and evaluating the effectiveness of the pedagogical theories that they utilize (Rahman & Ramos, 2003). Although grades are used to evaluate students’ work, these only represent average estimates of the overall quality and provide negligible information about students’ strengths, weaknesses, or ways to improve. Analytical assessment, which can be structured through different tools such as using detailed rubric and written audio or video feedback, can reveal the exact concepts a student finds challenging. In ODeL, software systems (e.g., Learning Management Systems) are often used to support flexible learning and to perform a variety of functions such as designing assessment tools, marking rubrics, assessment feedback, and timetable of course activities, among others.

In assessing student learning across courses, summative and formative assessments would address how well students are learning during the progression of a particular programme or over the years of offering at the college or faculty. Evidence is sourced from the embedded work in individual course, student e-portfolios that assemble student work in a number of courses, projects, or even relevant external exams such as a licensure exam. Given the requisite formats and data, students may be able to collect the progress of their own learning across courses (Miller & Leskes, 2005). Assessment results are useful as summative or formative feedback for students to understand their progress over time. Likewise, these are also used as feedback to programme faculty on how well the students are meeting the goals and expected outcomes.

In assessing courses, data gathered from course assessments could be seen as formative feedback for instructors to make adjustments in their teaching, or summative feedback to help the instructor or the committee in charge of the course to make informed planning strategies (Rahman & Ramos, 2003). Instructors and committees are expected to set expectations, establish common standards for multi-section courses, understand how the course fits into a cohesive learning continuum, and use the evidence to improve teaching in the context of ODeL environment, including course design.

Programme level assessment is defined as “the systematic and ongoing method of gathering, analyzing, and using information from various sources about a programme and measuring programme outcomes in order to improve student learning as it aligns with the institution’s mission and goals” (University of Central Florida, 2008). Assessment at this level intends to focus on the learning, growth, and development of the students not as individuals, but rather, as a group. Similarly, programme level assessment looks into whether the programme fulfills its aim for the curriculum by providing an understanding of what the graduates know, what they can do with this knowledge, and what they value as a result of this knowledge. 

In an ODeL context, a number of factors (e.g., selection and/or production of assessment instruments, conditions for assessment, supervising and authenticating, and managing procedures for possible revision and re-assessment) should be considered when doing programme level assessment. Particularly useful and important is the end point data as a summative gauge to determine how well the programme achieves its goals and objectives. Evidence of student learning from multiple sources (e.g., at admission, programme midpoint, or at the end of the programme) and levels also contribute to programme level assessment. In some ODeL institutions, selected assignments are re-scored (second reading) by programme faculty to assess the general education programme’s success in achieving institutional goals, such as communication skills, critical thinking, and ethical responsibility, among others.

Finally, institutional level assessment is undertaken either for improvement of internal operations and management (i.e. resource allocation, faculty hiring, faculty and staff development, collaborating, and networking) or fulfilment of external accountabilities. Results of the former can often also serve the latter purpose. According to Morgan and Taschereau (1996), assessment at this level is a comprehensive approach for profiling institutional capacity and performance as various factors come into play in this level of assessment. These factors are: 1) “the forces in the external environment (administrative and legal, political and economic, social and cultural)”; 2) “institutional factors (history and mission, culture, leadership, structures, human and financial resources, formal and informal management systems, and an assessment of performance)”; and 3) “inter-institutional linkages” (Morgan & Taschereau, 1996).

At institutional level assessment, a significant body of evidence coming from multiple sources are necessary to address questions such as: “What do the institution’s academic programmes add up to in terms of student learning?,” “How well are the goals and outcomes for student learning achieved?,” and “How can institutional effectiveness be reported to the external stakeholders?” (Miller & Leskes, 2005).

These evidence can come in the form of documentation that show how well the students are meeting institution-wide goals. Summarized data aggregated by course, by programme, or by student cohort, supplemented by results from relevant exams (admissions exam, licensure, etc.) would largely contribute to the institution-wide assessment. Hence it is important for institutional leaders to work in close collaboration with the faculty and staff, student affairs, and other professionals when designing a comprehensive programme of institutional assessment that address the following concerns: 1) the mechanisms being used to gather evidence of student learning and to provide feedback to students; and 2) the system of communication made to provide students’ access to needed resources and others.

Assessments in ODeL are no less valid and reliable than assessments in the traditional setting. The assessment instruments are expected to be appropriate and the criteria clearly defined and capable of generating evidence for the objectives and outcomes to be assessed. However, identifying or creating assessment tools for O-DeL can be quite challenging, considering that assessors are not in direct contact with the student whose performance is the most valid method of assessment. To ensure the reliability of the flexible learning provision of ODeL, assessment decisions must be as consistent as possible, subject to some internal checking, both in the course and programme levels. Moreover, appropriate training of the people involved in the assessment should be required (SQA, 2000). Support for student assessments should also integrate a variety of approaches such as flexipacks, scheduled tutorials, electronic-support modes, tutor-marked assignments, supervised workplace, or testing center assessments. With the introduction of information and communications technology (ICT) to student assessment, flexible learning programmes, which involve educators from both the conventional and ODeL modes, find enhanced flexibility, authentication, and security. Issues on communication, delivery, and administration, are also addressed. However, the concern with flexible learning environments is it has been constantly changing and responding to global initiatives and the advances in technology.

Quality has always been a concern of higher education institutions. UNESCO (1998) has referred to quality in higher education as a “multi-dimensional concept” that encompasses all academic functions and services ranging from “teaching and academic programmes, research and scholarship, staffing, students, facilities, faculties, and services to the community and the academic environment”.

Quality assurance in higher education, on the other hand, has been characterized as a continuous improvement of academic processes. It is the means towards assessing the potential of educational institutions in effectively performing and delivering these functions and services (Friend-Pereira, Lutz, & Heerens, n.d). Conducted either through internal self-evaluation or an external review body (UNESCO, 1998), quality assurance uses a set of agreed upon standards which may vary in terms of scope, depth, emphasis, and review components (Southard & Mooney, 2015).

In the literature, several quality assurance frameworks have been articulated. Often, these are associated with assessment and accreditation. Currently, there are several accrediting bodies, institutes, consortiums, and trade associations established especially for distance learning in higher education worldwide (Southard & Mooney, 2015). With the common goal of ensuring quality in distance education, they take initiatives in developing sets of standards, criteria, guidelines, or benchmarks which can address the uniqueness brought about by distance education in higher education.

For higher education institutions in the ASEAN context, the ASEAN University Network (AUN) developed the AUN-Quality Assurance Framework which focuses on assessment at the programme and institutional levels (ASEAN University Network, 2016). Programme level assessment constitutes a total of 15 criteria which include: Expected Learning Outcomes; Programme Specifications; Programme Structure and Content; Teaching and Learning Strategy; Student Assessment;Academic Staff Quality; Support Staff Quality; Student Quality; Student Advice and Support; Facilities and Infrastructure; Quality Assurance of Teaching and Learning Process; Staff Development Activities; Stakeholders Feedback; Output; and Stakeholders Satisfaction. Institutional level assessment, on the other hand, constitutes a total of 22 criteria which include the following: Vision, Mission, and Culture; Governance; Leadership and Management; Strategic Management; Policies for Education, Research, and Service; Human Resources Management; Financial & Physical Resources Management; External Relations & Networks; Internal Quality Assurance System; Internal & External QA Assessment; IQA Information Management; Quality Enhancement; Student Recruitment and Admission; Curriculum Design and Review; Teaching and Learning; Student Assessment; Student Services and Support; Research Management; Intellectual Property Management; Research Collaboration and Partnerships; and Community Engagement and Service.

In the field of open and distance learning, the Asian Association of Open Universities (AAOU) Quality Assurance Framework has been developed (Belawati & Zuhairi, 2007; Darojat, et al, 2015). The framework sets out important guidelines for each of the ten strategic issues identified in the distance education system: Policy and Planning; Internal Management; Learners and Learners’ Profiles; Infrastructure, Media, and Learning Resources; Learner Assessment and Evaluation; Research and Community Services; Human Resources; Learner Support; Programme Design and Curriculum Development; and Course Design and Development.

A cursory glance at these existing quality criteria shows that there are obvious commonalities and overlaps when categorized along the levels of assessment (Figure 2). Most quality assessment criteria are concentrated at the institutional level suggesting the importance of institutional leadership and the organization’s commitment to ensuring quality.

And while these have been shown to be sufficient for most teaching and learning situations, the evolution of higher education institutions coupled with technological advances highlights the need for quality considerations appropriate in a networked and digital environment.