Getting Results, Not Testing: Assessment in an Online Setting | Benjamina Paula G. Flor

0
(0)

This chapter submits that the use of traditional pen-and-paper tests in online learning is counterproductive. The physical absence of the teacher is opportunity enough to cheat or circumvent the process of learning. Twenty-first century learners, especially online ones, are independent and should not be monitored. More authentic assessments are necessary for students to generate knowledge products that emanate from their experience which made sense in formal education. As adult learners, they should be left to explore their creativity. This chapter is an attempt to showcase authentic assessment methods used in DEVC 202 (Development Communication Concepts and Approaches). These approaches include writing a journal, oral examination, and reflection synthesis piece. Such methods enjoin students to dig through their instructional materials deeply without force, but make them realize what their learnings are. Students are taught how to learn rather than what to learn. In the end, students begin to nurture their higher order thinking skills using instructional materials provided these are substantiated by their rich experiences as professionals in whatever field in which they are engaged. Giving the chance for students to think for themselves not only stimulates critical thinking but allows them to “go out of the box” to make sense of what has been learned not taught. Thus, the study assumes that online teachers should consider more authentic assessments methodologies given the changing profile of online learners. For online teachers, results – not scores – should matter.

 

An online teacher is faced with issues and challenges in providing quality education. This is because spatial separation of the teacher from the learner implies cheating or the propensity to ask for help in doing the course requirements especially when sitting for an examination. Given the nature of online education, a pen-and-paper assessment becomes counterproductive on the assumption that rote or lower order thinking skills have little value in addressing the needs of the 21st century learners.

 

Conventional examinations are easier to formulate and check. Questions are entirely dependent on the whims and caprices of the teacher and may not be anchored on the course objectives. While the method could be an expression of academic freedom, it limits students’ ability to express themselves. Essay questions could address this limitation but difficulty in checking poses a problem although there are now application softwares that can do this.

 

With a class size of 140 or more students per semester, the teacher may think twice about administering essay examinations. The past seven years with the UP Open University made me consider changing the way of assessing students’ performance in DEVC 202 (Development Communication Concepts and Approaches).

 

It is a fact that examinations can make students’ life dreadful. As Anderson (2003) points out, assessment formats largely rest on teachers’ decision-making. The teacher has the liberty on what to ask, how to ask, or what students should do in classroom assessment. Moreover, in conventional classrooms, numerical grades are mostly the basis to rate students’ performance. But how valid are examination items to warrant such grades? One might say using the “Table of Specifications” can guarantee the learning strands that need to be learned. Standards have been developed per subject and a rubric that goes with it as bases for assessment.

 

Initially, DEVC 202 requires students to complete three Tutor Marked Assignments (TMAs), write their reflections on the assigned readings through a journal, participate in three discussion forums where students have to react to each other’s post, and accomplish a written examination. The thought of formulating, administering, and checking the written examination was daunting. On the other hand, if there will be no examinations, there is no way of knowing the student’s take away from the course. Thus, a more authentic form of assessment came to mind in the form of a journal. However, since this is not to be submitted, there is no opportunity for the teacher to assess.

 

According to Burke (2009) authentic assessment involves three concepts: authenticity, assessment, and evaluation. Authenticity means real, genuine, or true to life. In terms of pedagogy, assessment is the process of gathering quantitative and/or qualitative information and organizing them into an interpretable form for easy judgment and decision-making. Evaluation, on the other hand, is a systematic process of judging the worth or merit of something (personnel, program, or policy) based on appropriate criteria that are clearly defined and articulated. Thus, we assess student performance, not evaluate.

 

Burke describes authentic assessment as not assembling disconnected pieces of information but rather doing a meaningful task that embeds content to provide connection between school-based ideas and real world experiences. It requires students to apply or synthesize information learned and to perform or demonstrate their understanding of the material according to specific criteria. Authentic assessment has the following features: emphasis on metacognition and self-evaluation; learning that transfers; positive interaction between assessor and assessee; quality products and performance; clear standards and criteria of excellence; and meaningful performance task.

 

In online learning, learners are autonomous and independent. They talk mostly to their learning materials.

 

According to Wedemeyer (1969) as cited by Moore (1973), the traits of autonomous learners are as follows:

  1. like to plan ahead – a day, a week, a month – and longer;
  2. usually stick to a plan, modifying it as they go along, but never abandoning a plan without improving it to serve their convenience and to help them “keep at it”;
  3. organize their lives to make the best possible use of time, the most critical ingredient of successful independent study;
  4. realize they can’t start a new activity (learning) without giving up something else that formerly took the time now set aside for study;
  5. enjoy reading, writing, listening, and discussing;
  6. have open minds to learning new things;
  7. enjoy questioning, testing, and analyzing;
  8. are not afraid of being different;
  9. likely to form generalizations, look for principles, and find the basic structural ideas in any subject;
  10. have developed skills in note taking, remembering, and relating; and
  11. work cooperatively with others, but enjoy being “on their own” in learning.

 

These characteristics of learners do not develop overnight but should be taught especially under an innovative education delivery mode.

 

Given the characteristics of authentic assessment, I added two more requirements in DEVC 202: authentic assessment through a 15-minute oral examination online via Skype and a Reflection Synthesis Piece (RSP) in addition to the journal described as follows:

 

The Journal. Students are expected to keep a record of learning for the entire semester. Any notebook will do, but I recommended a hardbound record book. This journal is where students keep their notes and execute their activity sheets. They need not post their responses in the portal as indicated in some of their activities. Their assignments, posts, and dates of postings in the discussion forums should be noted down in their journal. They will bring this journal during the Finals as a “passport” to take the exams. Being in a digital environment, students were allowed to use WordPress or blogging to record their reflections. Uniform Resource Locators (URLs) instead were sent to me for information and access. The journal is for the students to keep as future reference or repository of knowledge gained from the course. The student must take this exercise seriously, which constitutes 20% of the total markings.

 

Oral Authentic Assessment (AA) via Skype. Students have to sit for an oral examination based on a specific schedule on the following dates: 7, 14, 21, 28 or four Saturdays during the month of October. The semester started on 9 August and would end on 11 November. This implies that students must have gone through all the materials before they sit for the AA. Questions were formulated based on the objectives of the course, mostly applied but theoretically – sound. Students were given 15 minutes each for the AA. They were to choose a number between 1 to 30. The AA will start from 8AM to 5PM. In case the student will not be available on the designated date and time, they have to notify me one week ahead of schedule so I can make another time for them especially if there is a time difference since some of our students are offshore (USA, Canada, Italy, Qatar, etc). Students are instructed to invite me using my Skype ID. Since there could be many using the same skype ID, they have to make sure that I accept their invitations prior to sitting for their AA. This constitutes 25% of the grade.

 

Reflection Synthesis Piece (RSP). This is another form of authentic assessment given as an end-of-term project. It illustrates and projects student’s learning connected to real life experiences or one’s practical understanding of the course. The RSP paints a picture of student’s synthesis of the course and at the same time focuses on one’s passion or what one has bias about. It gives the teacher an idea on what students prioritize in life and an indication of how substantial the learning was. RSP highlights in less than three minutes a video, a haiku, a poem, an infographic, a short story, a plug, a poster, a dance, a painting, a comics or any communication material that conveys a personal take on the course. The RSP can also serve as an instructional material for lessons in class. The same can be shared with other students who are not in the same class and can be used as exemplars to explain the course in a simpler, practical, and concise point of view. RSP constitutes 30% of the grade.

 

While a Final Examination is allocated some points (10%), the Faculty-In-Charge (FIC) reserves the right to waive this requirement depending on the collective performance of the class especially in the discussion forums. Students are expected to respond and comment about their classmates’ posting. In this manner, students learn from each other. Final examination will be administered at the designated Learning Centers (LCs). There are two versions of the Final Exams: one for in-country students and the other, for overseas students. Overseas students shall take the Finals on a mutually agreeable, prearranged time, and place. In-country students who, for unavoidable reasons, cannot take the exam on the given date will be allowed to take the overseas version instead also on a mutually agreed upon date. The finals constitute ten percent (10%) of the grade. The tutor or proctor present shall examine the DEVC 202 journal before distributing the examination sheet.

 

In sum, authentic assessment measures constitute 75% of the total grade. This implies that students have to comply with all these requirements in order to pass the course.

 

Since the purpose of this chapter is to describe how authentic assessment measures are conducted in online learning specifically in DEVC 202, it will attempt to explain step-by-step how each assessment tool is carried out and rated. Rubrics are usually used as metrics to measure qualitative performance. However, in these authentic assessment measures, impact on the senses is central in gauging the unique crafting of knowledge products that students have produced to illustrate their learning. This could sound subjective, but in the long run what is being measured are experiences and realizations that are personal in nature. Regardless of the field, students who have seriously read, analyzed, and synthesized the readings would have realizations that could spontaneously flow out of their minds and are reflected in their spoken or written words, drawn pieces or creation of outputs that give a reader goose bumps, make her or him shed tears, or affect her or his perception on or attitude towards development issues. Such effects show how students have turned their learning into something that they can be proud of and something that they can put in their portfolios.


In the context of DEVC 202, oral examination is labeled as the AA, a 15-minute encounter of the teacher with the learner via Skype, the first and only virtual meeting for the semester. With 140 or more students, 15 minutes of engagement is equivalent to 495 minutes or 8 hours per day for 33 students, and at least four days to accommodate everyone.

 

Step-by-step procedure:

 

  1. Identifying the schedule

 

  • – Set the dates for the AA at the start of the semester and indicate this in the Course Guide. UP Open University examinations are usually scheduled on a Saturday but since there may be members of the Seventh Day Adventist, exams can be conducted at 7:00 AM or at some other agreed upon time. The reason for fixing the date early on is for students to program their schedules and be ready at the designated time since they will only be given one slot to sit for the AA. In case they lose their turn, they will have to wait for a vacant slot should the assigned student fails to show up for the AA.

 

  • – Prepare a time schedule for each student. Generally, order of examination is done alphabetically for easier tracking.

 

  • – Provide Skype ID and ask students to invite the FIC at least a week before the AA schedule. It would be better if the invitation can be done earlier to avoid problems like not receiving acceptance of request due to wrong Skype ID details.

 

  • – Allow time negotiation for offshore students in case of time differences.

 

  • – Allow students who missed their slot to sit down for the AA whenever they catch you online, which means the FIC is ready to receive a ping.

 

  1. Looking presentable on camera

 

  • – Wear make-up even early in the morning; it indicates that the FIC means business.

 

  • – Wear an attire that communicates professionalism, thus simulating a conventional classroom setting where teachers have to look presentable.

 

  • – Put on an attractive top that looks good on screen; since the exam is mostly done sitting down, the FIC can wear shorts or loose pants or skirt or something comfortable for the eight-hour exam.

 

  • – Make sure that the room is well lit so that students can see the FIC clearly.

 

  • – Ensure that Internet connection is stable and the microphone is functioning well.

 

  • – Look cheerful, warm, welcoming, and accommodating; the aura can ease tension and reduce anxiety in meeting the FIC for the first time.

 

  1. Setting the tone or breaking the ice

 

  • – Greet the examinee warmly and ask how they wish to be addressed (by full name or nickname) and the correct pronunciation of their name (i.e., “How do I call you?” “How do I pronounce your name?”)

 

  • – Ask for their location (“Where are you now?” Normally, the response would be “at home” or “at office”)

 

  • – Follow-up with finding out where they live (“Where do you live?”)

 

  • – Let the conversation be as friendly as possible for the next two minutes to avoid intimidating the examinee.

 

  • – Ask if the examinee has her or his journal with her or him.

 

  • – Ensure that the examinee feels at ease before asking the first question.

 

  1. Asking the question

 

  • – Inform the examinee on the number of questions to choose from (e.g., “I have 30 questions”).

 

  • – Ask the examinee to pick a number between 1 to 30 (e.g., “I’d like you to pick a number between 1 to 30”). Usually, examinees choose a number that corresponds to their birthday or a lucky number like 7 or 9. Sometimes, three consecutive students may pick the same number, which implies that students would have shared the question. When in doubt, give a different question.

 

  • – Read the question and write it on the chat box for the examinee to read and think about.

 

  • – Wait for clarifications or for the answers while observing the examinee’s face.

 

  1. Observing non-verbal cues

 

  • – Examinee starts smiling sheepishly or looking surprised: this means she or he is not prepared to answer the question.

 

  • – Examinee starts rolling her or his eyes: this means uncertainty about the answer.

 

  • – Examinee opens journal and says, “I know I wrote it somewhere.” this means examinee has written it down but could not easily locate it. Give time to look for the notes. This is not cheating since it indicates doing the homework by writing down notes and reflections in the journal.

 

  • – Examinee starts explaining and seems to be lost for words: provide some leading questions or situations that encourage her or him to continue.

 

  • – Examinee cannot answer the question: give another chance by picking a different number. This is no different from taking conventional examinations where some questions cannot be answered.

 

  1. Ending the conversation

 

  • – Ask another question; the answer to which should be written on the chat box. Cut and paste the response in the FIC’s excel file to document the response (e.g., “How would you define development communication in your own words? Write it on chat and send it to me now.”)

 

  • – Inform the examinee if she or he passed the AA. Passing the AA is like joining a beauty pageant, which makes it worth remembering. The exchange of information could be the longest 15 minutes of the semester.

 

  • – End with a big smile and words of encouragement.

 

  • – Ask examinee if she or he has any question for the FIC. If there is none, bid the examinee goodbye.

 

  1. Grading the performance

 

  • – When responses are not theoretically grounded but close to the concept, it means that the examinee knows the answer but cannot fully explain it. This should not be taken against the student. Follow-up questions can be given to elicit a more appropriate answer by relating it to the student’s experience. In this manner, she or he learns from the FIC and her or his experience as exemplars becomes memorable.

 

  • – In case the examinee is bluffing or not answering the question correctly, give the corresponding points.

 

  • – Remember that this is the first virtual face-to-face meeting. Certainly, the examinee feels nervous and could experience mental block.

 

  • – Conversations can lead to student’s desire to leave a good impression on the FIC where they would try to articulate the concepts as applied in real life.

Creating the RSP gives students freedom to express their thoughts, feelings, and learning through a knowledge product. This is a higher order thinking skill that mirrors what the students have learned in the process of engagement.

 

Step-by-step procedure:

  1. Describe the expected output in the Course Guide.
  2. Let the student decide on what knowledge product to produce (poem, song, dance, etc).
  3. Avoid showing exemplars or sample work done by previous students so as not to influence their creativity.
  4. Give students enough time to synthesize the course contents by analyzing their journal, re-reading their materials, and going back to their discussion forums.
  5. In grading a performance, go over the material. One could readily sense the student’s realization upon reading, seeing, or hearing the output. Getting goose bumps or crying while watching a video or reading a poem implies the students’ deeper understanding of the discipline. The choice of subject and framing reflects how they understood the concepts, foundation, and philosophy of development communication. These lend credence to the shaping of would-be development communication practitioners.

 

Here are some examples:

 

Richard Joseph H. Bulanhagui

Development Communication, Oh, mighty one!

Ever helpful and caring to everyone…

Never stops offering to the not yet done.

Escapes the hunger and the oppressed bun.

Oh, glorious! Oh, glorious! Devcom!

Where did you come from?

Amidst the problems and norms,

You built up the things that were worn.

From Quebral to Flor,

Who have opened the door.

To communication floor,

For humanity and the store.

I will never forget my journalism class,

Where I found out the development task.

In which I can be fully blast,

Development Communicator that surely would last.

Never will I forget my first immersion,

Seeing people suffering from malnutrition.

Saving money for damn foundation,

Of bag of rice and sardines donation.

To see the world perish is not that I want.

I want to seek help for the poor and the blunt.

I want to scatter light to those dim in front.

I want to bring it better to the life of the peasant.

Maybe I am not a great harvester.

Not a great science or development broadcaster.

But as long as I am living, to this roller coaster.

I will bring harmony to the poorest dweller.

May this hope of peace never last.

The advocacy of the environment that shall to pass,

The conflict issues of the tribal wars,

I hope this would end before I’ll be buried in grass.

Devcom 202 is a great starter.

For the aspirants of a fruitful master’s.

This will mark the end of semester.

Where I will leave this poem for the Alma Mater.

 

“Pagtuklas”

Capili, Kristen Amaris D.

Hindi niyo ba napapansin na ang tanging pagkakaiba ng mga salitang bulong, tulong, at sulong ay nasa simula’t nag-iisang letra?

Na ang kusa at dusa ay tila magkatunog ngunit labis na

magkalayo ang diwa?

Na ang kubo at subo, na halos iisang bigkas ngunit magkaibang kahulugan ay kolektibong nangangahulugan na ang bawat mamamayan ay may pangtaong karapatan.

Sa mundong nagkakaubusan ng pasensya’t katarungan at ng perang hindi malaman kung saang lupalop ang pagkukuhanan,

Ano at sino’ng naghihirap para sa kapakanan ng ating kapwa at ng Inang Bayan?

Kung malalim ang pagnanais nating umahon sa kahirapan, sa kabuhayan man o usapang kalikasan at kalakasan.

Maliban sa tapang na araw-araw nating kailangan, malasakit na walang pag-aalinlangan at wastong gamit ng utak ang dapat

nating gawing puhunan.

Matapos ang pag-iisip na masinsinan, minabuti kong piliin na itong landas ang tatahakin. Dahil sa kasalukuyan kong araw-araw na pakikipagsapalaran, hindi ko maramdaman ang katuparan

mula sa aking kinauupuan.

Komunikasyon: isang solusyon tungo sa pag-unlad; isang bokasyong pinaniniwalaan kong walang katulad.

Mahirap, ngunit magsisikap. Mahirap, ngunit may mabuting hinaharap. Mahirap, pero ito ang aking pangarap: Maging susi sa pagtupad ng mga pangarap ng mga kapwa nating naghihirap.

 

“What It Is Not”

Lorelli Monica Diaz

I

Now that we’re almost at our semester’s end Development communication – how do we comprehend? Here is a succinct poem to explain what it’s ought. To understand what it is, let us see

what it’s not.

II

Dev comm as we call it does not seek for fame. Image-building and publicity are simply not its aim. Stimulates public knowledge and understanding of planned change

And uses appropriate media tools to broaden its range.

III

Mass communication alone, it should not be considered as.

Various methods it explores so it fulfils what it does.

A short film, a dance, not an approach to be discarded

As long as the message is well-delivered, received and regarded.

IV

Dev comm does not have a “boss” it needs to impress

Rather societal problems that it needs to address.

The information and the receivers matter the most

And how the messages improve lives from coast to coast.

V

Development communication is not merely a profession. It is the offspring of commitment to growth and of true compassion.

It’s a calling, it’s hard work, an intervention and a process

That is ultimately aimed at rousing societies towards their progress.

 

“THE DEVCOM 123”

Ariesha Faith Dimaano

One,

We only have one world to live,

And I hope we don’t only have apathy to give,

Problems as poverty, unemployment, malnutrition, inequality,

There are much much more but would we want plenty?

All these realities point to underdevelopment, Molenda and Di Paolo taught us what problematique meant,

The goal is to write all situations and identify,

Link everything to each other and keep asking why.

As what “the Moth and Development” implied,

Development is a process and the struggle to achieve it is tight,

It has to be like a two-way street,

Where the DevCom worker and its audience would always have to agree and meet.

Two,

There are two words in this field we

That is development and communication,

The former seeks to better the society’s current fate,

The latter bridges the gap with the power to inform, influence, and educate.

Defining communication comes in theories and models,

Some from Shannon, Weaver, Berlo and Lasswell,

ut Kincaid’s convergence model is my personal favorite,

As it best highlighted information sharing to reach mutual understanding and agreement.

Communication is also expressed through all forms of media,

But it’s always important to choose what best fits the DevCom study and data,

Consider the barriers that come with it too,

Take it as a challenge and there’s something you can do.

Three,

DevCom has three major values,

It is pragmatic, value-laden and communication with a purpose,

As former Chancellor Samonte would put sense,

DevCom is “communication with a conscience.”

An important pillar in honing the future,

DevCom has been visible in other disciplines’ nature,

Take economy as an example,

Where much is information-based and shifts to networking

with people.

Dynamic and progressive, DevCom practitioners have areas of practice to dedicate himself,

All of which determines one edge,

To continue the journey for positive social change.

As an aspiring DevCom practitioner myself,

This passion has been put on hold on a shelf,

Now that I found my chance,

May I be able to take this stance:

If I really consider this as a calling,

May I be able to put meaning to all the learning.


These results are more significant compared to answering an examination because they leave a mark in the hearts of learners as they continue with the program. Out of 140, 102 submitted the RSP and took the AA. This is a very good indicator of less than 30% attrition rate. Admittedly, many students of the Master in Development Communication come from various disciplines such as nursing, education, information technology, law, etc. They do not have a full understanding of development issues nor how they could be catalyts of change. Going through the authentic assessment measures made them realize that development communication is a noble undertaking and that being a development communicator can make a difference in an erstwhile chaotic world.

References
Anderson, L. (2003). Classroom Assessment. New York: Lawrence Erlbaum Associates, Inc.

Burke, K. (2009). How to Assess Authentic Learning. Available at https://www.amazon.com/Assess-Authentic-Learning-Kathleen-Burke/dp/141296279X

Moore, M.G. (1973). Toward a theory of independent learning and teaching. The Journal of Higher Education, 44:9, 661-679. doi:10.1080/00221546.1973.11776906

Wedemeyer, C.A. (1969). How to earn college credit and a degree. Background paper for CEEB Handbook for Continuing Education in America. Unpublished paper, Madison, University of Wisconsin at Madison.

Suggested citation:
Flor, B. P. G. (2018). Getting Results, Not Testing: Assessment in an Online Setting. In M. F. Lumanta, & L. C. Carascal (Eds.), Assessment Praxis in Open and Distance e-Learning: Thoughts and Practices in UPOU (pp. 99-112). Los Baños, Laguna, Philippines: UP Open University

How useful was this resource?

Click on a star to rate it!

Average rating 0 / 5. Vote count: 0

No votes so far! Be the first to rate this post.

As you found this post useful...

Follow us on social media!

We are sorry that this post was not useful for you!

Let us improve this post!

Tell us how we can improve this post?

Getting Results, Not Testing: Assessment in an Online Setting | Benjamina Paula G. Flor

Leave a Reply