The University of the Philippines Open University (UPOU) is mandated to provide wider access to quality education as well as to contribute to the upgrading of the country’s educational system. Aside from offering relevant degree programs and continuing education courses through open and distance learning, the mandate is also carried out with the sharing of university created multimedia learning materials. As the UPOU espouses the philosophy of openness, the learning materials are accessed and used by the general public without a fee. While the materials are shared for free, quality assurance (QA) approaches are observed in each development, production, and publication phases/stages. In this chapter, QA practices are presented and future challenges are considered.
The University of the Philippines Open University (UPOU) was established to provide Filipinos, especially those living in areas far from University of the Philipines residential campuses, access to quality education that the country’s premier state university offers. For more than two decades, UPOU has pioneered the offering of degree programs and continuing education courses through open and distance learning (ODL) in the Philippines.
Aside from its degree programs and continuing education course offerings, UPOU also contributes to the further improvement of the quality of education in the country through the learning materials it continues to develop, produce, and share not only with its learners and faculties-in-charge (FIC), but with all types of learners, educators, and trainers of other learning institutions.
The UPOU produces an array of learning materials, both in text and multimedia formats. The production of which goes through processes to assure the quality and usefulness of the materials through its Multimedia Center (MC). The center provides know-how and technical assistance to faculty offices, faculty members, and other units and programs of the university for the development, production, and distribution/publication of multimedia learning materials.
Quality assurance (QA) entails the management, organization, and implementation of evaluation procedures that meticulously examine the quality of a facility, service, and/or product (Almedro & Silveira 2018; Bucki 2019). Harman 2000 (as cited in Belawati & Zuhairi 2007) said that QA processes for higher education institutions, serve to “ensure achievement of quality outputs and further quality improvements” (p. 2). The QA processes focus on creating products with built-in qualities, this is made possible through monitoring the quality in every step of the production process and not when the product is already produced. Almendro and Silveira (2018) said that with constant monitoring, mistakes or oversights are sorted out and remedied right away and not when the product has been produced. At the MC, monitoring of quality in every step of the production process is highly recommended and adhered to. The MC collaborates with the Faculty Offices and the Office of Academic Support and Instructional Services (OASIS) in the development and production of multimedia learning materials. The UPOU Faculty Offices— Faculty of Education; Faculty of Information and Communication Studies; and the Faculty of Management and Development Studies—are in-charge of the development, offering, and delivery of degree programs and courses. The OASIS, on the other hand, is the office responsible for the development of instructional/learning materials in text formats.
The development of multimedia material goes through three major phases— (a) pre-production, (b) production, and (c) post-production (Figure 1).
Figure 1
Creation cycle of open educational resources (OER) in multimedia format
Pre-production Phase
This phase consists of two stages, conceptualization and design. Under the conceptualization stage, the course proponent and/or the FIC of the course prepares a proposal for the production of the multimedia learning material. The proposal needs to be reviewed, approved, and endorsed by the Program Chair to the Faculty Dean. The MC shall only act on concept notes/proposals that have been approved and endorsed by the Faculty Dean. At the conceptualization stage, the content creator shall have chosen among two options with regard to how the learning material shall be produced:(1) the content creator can opt to do production on his/her own, or (2) the content creator can tap the MC team to provide production assistance. For option one, the content creator can borrow production equipment and request the use of studio facilities from the center. Under the second option, production meeting/planning/orientation for all people involved shall be done prior to the actual shoot. Among the topics for discussion are human and non-human resources requirements for the OER to be produced, (i.e., talents; audio, video, and graphics requirements; location shoot; set design; among others). Upon receipt of the approved proposal, the design stage shall commence with the MC convening the Quality Circle, which consists of the following:
a. FIC who shall serve as content creator and/or subject matter
specialist;
b. Multimedia specialist;
c. Instructional designer; and,
d. Other members of OER development team (i.e., universal accessibility
expert, gender specialist)
The multimedia specialist and the instructional designer can be one person. This person shall assist/guide the content creator in choosing the appropriate format to be used and in developing the script/storyboard. Even at the design stage, evaluation is already commenced. The following aspects, among others, are looked into: attainment of objectives of the multimedia materials; accuracy of the content; language, concepts and terms used; gender fairness; and, universal accessibility. Revisions on the script/storyboard are carried out if deemed necessary.
Production Phase
Development of the multimedia materials/OER pushes through upon approval of the script/storyboard. At this stage, revisions in the script during the shoot or development/production of the materials may occur. As experienced by the center, there were times that a certain portion of a learning material cannot be executed as written in the script. Improvisation shall be done to further enhance multimedia material’s learnability and usability.
Post-production Phase
There shall be three stages under the post-production phase. First is the production QA, wherein the alpha version of the multimedia material shall be reviewed by the members of the quality circle and the FIC/content creator. At this stage, other faculty members who are teaching similar course/ courses can also be requested to evaluate the material in terms of content, learnability, and usability.
Revisions on the alpha version shall be done based on the comments/ recommendations of the FIC, quality circle, and other reviewers from within the university. The revised material, beta version, shall then be presented to the FIC and quality circle for approval. When approval is secured, the multimedia material shall be curated and published in the UPOU Networks, the second stage under this phase. The UPOU Networks is the online repository of the multimedia and text-based teaching and learning materials of the university.
Once uploaded in the UPOU Networks, the material shall be opened for peer and public reviews. This is the third stage under the post-production phase. Referred to as the publication phase, QA is just as important in this phase. Results from this evaluation stage are essential as the reviews, comments, and suggestions shall be coming from the users/learners. The valid comments shall be the basis for revisions to be done on the multimedia learning materials.
The center uses a continuous process of development/redevelopment. As to what stage of redevelopment a produced material shall go through, depends on the extent of revisions that have to be done. Revision can go back or pick up from any of the phases and/or stages.
When evaluating the quality of the produced multimedia materials or OER, the MC looks into different aspects, which can be categorized into two groups: (a) content and its presentation; and, (b) the technical aspects.
For content and its presentation, the following questions serve as guide in evaluating the material:
1. Does the content and treatment of the information/knowledge presented can make learners achieve the material’s learning objectives?
2. Does the treatment and fluidity of the presentation of the information make it easier for the learners to understand the topic presented?
3. Is the information presented accurate and up-to-date?
4. Is the language and/or words used make it easier for the target group of learners to grasp the content of the material?
5. Does the material promote equality and inclusivity (i.e., has no bias against gender, culture, religion, disability, socio-economic status, among others).
6. Are the graphics, video clips, music, and similar multimedia content appropriate for the topic/subject matter and for the target audience?
7. Is the length of the material right? Is it too long/dragging that bores the audience?Can the material be chunked to shorter visual or audio? Or is it too short that learners would wonder if the material has really ended?
In terms of technical quality, the following are looked into:
1) Quality of the video
Is the video clear enough? Does the treatment of the video clips vary to emphasize important details?
Are the quality of the video clips, animations, or texts included in the material make it easier for the learner to follow/understand what is being discussed or being presented?
Are the colors used appropriate, especially for the color blind?
Are the sizes of the videos, text, and graphics large enough for the audience to see?
Does the accompanying text in the video stay on screen, enough for the viewer to read and understand what was presented?
2) Quality of the audio
Is the voice-over audible?
Is the volume pleasant enough or it sounded more like a noise that makes listening, thus learning too, unpleasant?
Are the music and sound effects used appropriate to the topic being presented?
One of the undertakings of the university promotes the attainment of United Nations’ sustainable development goal number 4 (SDG 4)—that of ensuring inclusive and quality education for all and the promotion of lifelong learning —and the United Nations Educational, Scientific, and Cultural Organization’s (UNESCO) Learning for All: Guidelines on the Inclusion of Learners with Disabilities in Open and Distance Learning.
The university develops OERs or the “teaching, learning, and research materials in any medium that reside in the public domain or have been released under an open license that permits no-cost access, use, adaptation and redistribution by others with no or limited restrictions…” (UNESCO, n.d., para. 1). The term OER was developed in 2002 during UNESCO’s Forum on the Impact of Open Courseware for Higher Education in Developing Countries. Since then a community of OER authors and producers have emerged as well as establishments that are using OER in their teaching and learning activities. The popularity of its use is brought about by several factors, among which are the following:
• provides opportunity for collaboration, dissemination or sharing of knowledge, and innovation of educational contents (Bliss & Smith, 2017);
• allows faster transfer of “high-impact practices in pedagogy” (Plotkin, 2010);
• promotes digital literacy, a necessary skill in the current century, as it uses digital technologies to produce, share, and access OER (Geser et al., 2007);
• allows flexibility to choose course materials that can be used, modified, and integrated in teaching and learning (Geser et al., 2007);
• promotes student-centered approach to teaching and learning as the students are are not restricted to share their studies while at the same time utilize learning resources (Geser et al., 2007);
• saves time and effort of learning facilitators, as OER can be re-used, adapted, and/or customized based on the characteristics and needs of the course (Geser et al., 2007); and
• addresses specific needs of a user or a learner as it is adaptable (Bliss & Smith,2017).
As the UPOU espouses the philosophy of openness, the university-created OERs are uploaded and curated in the UPOU Networks website. Access and use of these materials by the public are without a fee, but while this is the case, QA approaches are observed in each development, production, and publication phases/stages to ensure that materials are accurate and usable. The development of QA practices for the development, production, curation, and reproduction/revision of OERs in the center are influenced by different practices, principles, and framework. Among which are the Principles of Universal Design for Learning (UDL), UNESCO’s Guidelines to Ensuring Inclusion and Equity to Education, Mayer’s Principles of Multimedia Learning, and the TIPS Framework of QA for OER.
Principles of Universal Design for Learning
The Universal Design for Learning (UDL), as implemented in UPOU, is a set of curriculum development principles that will give learners equal opportunities to learn. It is a blueprint that can be used to “create instructional goals, methods, materials, and assessments that work for everyone” and “provides flexible approaches that can be customised and adjusted for individual needs”(Center for Applied Special Technology [CAST], 2018, UDL at a Glance section).
The following are the principles of UDL (CAST, 2018):
• Provision of multiple means of representation (the “what” of learning). Learners differ in the ways that they learn, perceive and comprehend information presented to them.
• Provision of multiple means of action and expression (the “how” of learning). Learners differ in the ways that they can navigate a learning environment and express what they know.
• Provision of multiple means of engagement (the “why” of learning). Learners differ in the ways in which they can be engaged or motivated to learn.
UNESCO’s Guidelines to Ensuring Inclusion and Equity to Education
The UNESCO guidelines, developed in 2017, will ensure inclusion and equity to education and center on the United Nations’ SDG 4. The SDG 4 also promotes the development of education facilities, including educational materials, that shall “provide safe, inclusive and effective learning environments for all” (UNESCO 2020, Safe and non-violent learning environments for all: Trends and progress section, para. 2)—they should be child safe, gender sensitive, and disability friendly.
Mayer’s Principles for Multimedia Learning
Richard Mayer, a Psychological and Brain Sciences professor at the University of California, came up with 12 principles that can be followed to be able to create effective learning materials in multimedia formats (Mayer, 2009).
The multimedia learning principles are the following:
1) Coherence Principle
Only relevant and simple components should be incorporated in the learning material. Materials that do not contribute to the achievement of the learning goals should be excluded.
2) Signaling Principle
Learning material should contain cues that can highlight or draw the attention of the learner to the important information in the multimedia material.
3) Redundancy Principle
Only the combination of spoken words and graphics enhance learning. Adding text on-screen should be avoided as it may overwhelm the learner visually.
4) Spatial Contiguity Principle
Text and graphics when presented near each other enhances learning.
5) Temporal Contiguity Principle
Simultaneous presentation of words and pictures makes learning better than successive presentation of these components.
6) Segmenting Principle
Producing long multimedia learning materials should be avoided. Learning materials should be presented in a segmented manner as it allows learners to control the pace of his/her learning.
7) Pre-training Principle
Establishing prior knowledge or scaffolding, such as learning key terms, of students should be done before exposing them to the multimedia learning materials. As the content of the learning material, especially when the topic presented is complex, may overwhelm the learners.
8) Modality Principle
The use of the combination of narration and pictures enhances learning more than the combination of printed text and pictures.
9) Multimedia Principle
The combination of words (whether narrated or printed) and pictures enhances learning more than using words alone. However, the propositions of the redundancy and modality principles should be taken in consideration.
10) Personalization Principle
The use of conversational style of narration or lecture in a multimedia material enhances learning more than formal style.
11) Voice Principle
A narration done by a human voice promotes better learning than using a computer or a machine voice.
12) Image Principle
Adding a speaker’s image on screen does not necessarily enhance learning. It was suggested to avoid including one’s video when graphics and texts are being presented in the multimedia material.
However, it was pointed out that the last two principles are for further studies.
The TIPS Framework of Quality Assurance for Open Educational Resource
This TIPS framework was first developed in 2013 by Paul Kawachi, a professor of Instructional Design in Open University of China, formerly the China Central Radio and TV University. Then in 2014, he came up with version 2.0. Both versions were published as OERs by the Commonwealth Educational Media Centre for Asia (CEMCA). TIPS stands for the four groups of QA criteria that teachers may adapt to assure quality of the material they will create or will choose from the various OER available on the Internet.
These four groups are as follows:
1) T — teaching and learning processes
2) I — information and material content
3) P — presentation product and format
4) S — system technical and technology
The first set of criteria under teaching and learning processes look into the following aspects concerning appropriateness of the resource for the students with whom the OER shall be shared: including usefulness of the resource; teaching approach to use; appropriateness of the language/word/ manner of delivery; potential of the OER to motivate the students to learn, be engaged, and to be more critical of the information shared in the material; localization of the information contained in the material or suitability with the realities of the learners.
Under information and material content, the following should be evaluated: the accuracy, reliability, authenticity, relevance, and up-to-dateness of the content; promotion of inclusivity, non-biasness, and equality; contextualization of the information; length of the material; usability of the material for other courses; and, availability of other sources to enrich the subject matter presented in the material.
For presentation product and format, the material should have an open license, preferably one that would allow reuse and remix of the material; portable, easy to access and share; has good quality audio, graphics, video and colors that would stimulate learning; and, can be used off-line or has mobile format.
With system technical and technology, the OER should have metadata tags to make the material easier to find and identify, thus, should include information on duration, format, file size, and difficulty level, if any; created on open source software and can be accessed or opened in various platforms and has a date of production and/or revision.
The MC’s QA framework had been presented to several faculty and affiliate faculty members of UPOU. There were three common concerns raised when the framework was presented. First is the tediousness and time-consuming process to develop the multimedia learning materials, though many had acknowledged the importance of going through each stage as such would ensure the quality of the learning materials. Second is the need to learn knowledge and acquire skills for the development of multimedia learning materials, from scriptwriting to actual shoot/recording/production. Last, is the access to equipment and technical assistance needed for the production.
To deal with these concerns, the center had started and shall continue to hold training programs needed for the development of multimedia learning materials. In the past years, training on and presentations on different topics had been done, such as use of applications/software that can be used for learning materials production; on-camera presentation skills of the content creator and/or talent; videography, among others. There were also presentations on the different services that the content creators can avail of from the center—proposal development, scriptwriting, instructional design, use of various audio-visual (AV) equipment and software/applications, and provision of recording studios.
Continuous upgrading of production hardware and software are also being done as well as construction of additional recording studios.
The development of learning materials is a continuous process. There shall be other challenges that may arise. To address the challenges, developers/ content creators, members of the quality circle, and of the production team also need to continuously upgrade their knowledge not only on the subject matter being presented, but as well as on pedagogy, and on new ways of presenting information through multimedia.
Another consideration is to formalize the set of guidelines presented here.
The guidelines are there, but they are not yet fully applied and adapted by everyone, especially when the planning, producing, and publishing their own educational resources for their respective online courses did not pass through the center nor through the OASIS.
Styles, formats, presentations, and treatments, in the future may evolve to something that may be altogether different in the ways information and knowledge are presented and shared at this time. The center shall be developing not only video and audio/podcast materials, but eventually will sink its teeth in the development or production of augmented reality (AR) and virtual reality (VR) materials to be able to keep-up with the evolving characteristics and learning needs of learners, the circumstances that they are in, and the advancements in information and communication technologies. But, as the center delves into the development of new materials with new treatments, the implementation of the QA process should never wane. It should be upgraded and further improved. To do this, the center should not only promote continuous capability and capacity building of its present manpower, but should also expand the number of its staff members with various expertise concerning OER development and tap the expertise of the university’s faculty members and research/extension personnel with regards to not only development of resources, but as well as pedagogy, instructional design, and research undertakings.
Belawati, T., & Zuhairi, A. (2007). The practice of a quality assurance system in open and distance learning: A case study at Universitas Terbuka Indonesia (The Indonesia Open University). International Review of Research in Open and Distributed Learning, 8(1). https://doi.org/10.19173/irrodl.v8i1.340
Bucki, J. (2019, July 02). What is quality assurance? The Balance Small business. https://www.thebalancesmb.com/definition-of-quality- assurance-2533665
Bliss, T.J., & Smith, M. (2017). A brief history of open educational resources. In R.S. Jhangiani & R. Biswas-Diener (Eds.), Open: The philosophy and practices that are revolutionizing education and science (pp. 9–27). Ubiquity Press. https://doi.org/10.5334/bbc.b
Center for Applied Special Technology. (2018). Universal design for learning guidelines version 2.2. http://udlguidelines.cast.org
Chan, L., Cuplinskas, D., Eisen, M., Friend, F., Genova, Y., Guédon, JC., Hagemann, M., Harnad, S., Johnson, R., Kupryte, R., La Manna, M., Rév, I., Segbert, M., de Souza, S., Suber, P., & Velterop, J. (2002). Budapest open access initiative declaration. https://www.budapestopenaccessinitiative.org/read
Geser, S., Salzburg Research, & EduMedia Group (Eds.). (2007). Openeducational practices and resources: OCLOS Roadmap 2012. https://www.olcos.org/english/roadmap/index.htm
Hodgkinson-Williams, C. (2014). Degrees of ease: Adoption of OER, open textbooks, and MOOCs in the global south. https://www.slideshare.net/ROER4D/hodgkinson-williams-2014-oer-asia
Kawachi, P. (2014). The TIPS framework version-2.0: Quality assurance guidelines for teachers as creators of open educational resources. Commonwealth Educational Media Centre for Asia (CEMCA). http://oasis.col.org/handle/11599/562
Plotkin, H. (2010). Free to learn an open educational resources policy development guidebook for community college governance officials. Creative Commons. http://wiki.creativecommons.org/Free_to_Learn_Guide
United Nations Educational, Scientific and Cultural Organization. (n.d.). Open educational resources.https://en.unesco.org/themes/building-knowledge-societies/oer
United Nations Educational, Scientific and Cultural Organization. (n.d.). Safe and non-violent learning environments for all: Trends and progress. https://en.unesco.org/themes/school-violence-and-bullying/sdg4aprogress
Gelisan, L. A.,Mangubat, L. J. (2020). Producing Quality Learning Resources in Multimedia Formats. In Quality Initiatives in an Open and Distance e-Learning Institution: Towards Excellence and Equity (pp. 105-118). Los Baños, Laguna: University of the Philippines Open University
