TESTING & EVALUATION WORKSHOP - the best ever !

TITLE : TESTING & EVALUATION

VENUE : De Palma Hotel, Ampang

DATES : 18-23/06/08, 16-19/08/08, 13-17/12/08

Alhamdulillah, I’ve just completed a 120-hours Testing & Evaluation (T&E) course cum workshop. The main objective of this workshop is to provide a basic knowledge and practical exposure in the development of quality, valid and reliable test items. My 4-member team comprised of Mohd Asri (KB), Norhisham (BP), and Mohd Zulfikar (BP) and yours truly. Honestly, this has been the best course I’ve ever attended in my 20-odd years in service with KPTM.

So, what’s so great about this course/workshop? Now, I’m sure each and every one of us has in our own way an experience taking a test. Your reason for taking the test could have been for a selection, classification, intelligence, an aptitude or may be simply for a placement purpose.

But how do we know that a particular test item set has any quality? What about their validity and reliability? Are we not concern at all?

Simply put, TESTING is a procedure for gathering valid and reliable information in evaluating the mastery level of our students on certain selected topics. It is to see whether they have learned what they are supposed to learn through the course of study.

To ensure this objective is met, a good quality test must therefore be constructed. But what is a good quality test? Simply put, a test of good quality will test what it is supposed to test on. Extra care and caution should therefore be taken to ensure that the test items were not confusing or misleading, what more if it is out of topic.

ITEM DEVELOPMENT PROCESS

The very first step in constructing test items is to plan its content area and the skills required for the students to be tested. To ensure that the test to be constructed will have a high content validity, references to the Learning Objectives (LO) and learning outcomes of the syllabus must be made when developing the Table of Specification (TOS).

Once you’re done with the writing process, then follow up with a reviewing process to see whether:

• The items constructed are according to TOS?
• There any repeats of the same question?
• The contents and key of the items are of high validity?
• The distracters can really discriminate between good and weak students?
• The sentence arrangement, format and layout of the items were clear, easily understood and not confusing?

WHAT I’VE LEARNT FROM THIS WORKSHOP ?

Constructing a good test item is not as easy as most people would think. It’s not just about having the right format and layout, but also requires going through various process of planning, selection, analyzing, abiding to certain standards and guidelines. All these are to ensure that the test items we construct will actually test what it is supposed to test on.

The quality of a particular test depends on the quality of each individual item that we construct. Selection and construction of items for the final form should be based on their TOS and item analysis, focusing on the following criteria:

• Does the item matches the LO?
• Is there any wrong key? – whether the key used is confusing and/or incorrect?
• What about ambiguity? – look into discrimination power (Point Bi-serial) or the r-value of the items. Good students should be able to answer correctly, and weak students answer wrongly.
• Is there any blind guessing? – to see whether the examinees responded blindly.
• The difficulty level (P-value).
• What is the percentage of students who answer each item correctly? – from here we can know who/what is not good : either the lecturer, the student or is it the question itself ?
• Effectiveness of the distracters used – whether they are good/bad. Do they function as expected?

A test is considered reliable only when it measures what it is supposed to measure, and consistently too ! And one need to remember also that our test scores may contain error that may lowers its reliability. These errors could have been due to several reasons:

• The test taker – perhaps the candidate had a bad day like a cold/flu or an argument with someone special.

• The test itself – the questions were confusing/ unclear. Some students failed in their exams not because they are weak or poor in the subject matter but mainly because the question items constructed were of low quality – vague, misleading and blatantly confusing! We have to be fair to them.

• The test condition – there may be distractions (like noise or even unsuitable lighting or room temperature) in the exam hall that affect his attention and concentration level.

Frankly, before attending this workshop, issues on validity and reliability have not been my main concern. But now, they will definitely become my top priority whenever constructing the test items.

The Kuder-Richardson formula-20 (KR20) reliability coefficient used in our exercises showed that the less number of items the test has, the less reliable the test is. That means to say, to get a more reliable test, we have to construct more number of items to it.

To reduce blind guessing, we have to have more than 4 options for the Multiple Choice Question (MCQ) items. Example: if we have 4 options, 1 divide by 4 equals 0.25, and with 5 options, we have 1 divide by 5 equals 0.20. That shows the more options we have, the smaller the percentage of guessing it will be (ie. Only 20% possibility of guessing for an item with 5 options, as compared to 25% for a 4 options item).

Overall, I do faced a bit of a challenge to keep up with the momentum and continuity of this workshop as it was split into three parts, delivered on three separate sessions spanning from June to December 2008. Fuyooohh ! 120 jam tu beb !

Nevertheless, a very practical and activity-based approach used by our “Mr. Cool” T&E expert, Dr Quek Bong Cheang (OUM) was able to keep our attention and spirit high, right from the beginning till the end. His lectures were really an “eye-opener”, very relevant and practical indeed. Terima Kasih CikGu Quek - I like your style!

Finally, I would like to say my gratitude to the management of KPTM for giving me this valuable opportunity to undergo this workshop with Dr Quek.

In future, I hope to see more KPTM lecturers (or even the relevant Admin people) selected to attend this course. Like me, I am confident that the others can also gain a lot from it – at least the basic knowledge and skills in constructing test items with a lot better quality, validity and reliability level. And of course, God willing, at the end of the day our students will also benefit too... InsyaAllah.

Oldies but goodies

Yesterday I bought a Canon scanner (CanoScan LiDE 25) for RM190 at LowYatt. For that price tag, I thought it was quite a good buy I think. The idea behind purchasing the scanner was for scanning newspaper cuttings, caricatures, photos, and other materials which are relevant for my lecture presentations.

Anyway, the followings are some of the scanned products of my "old family photos" that was originally taken way back circa 1966 when I was growing up in Kubang Kerian, Kelantan ... just to share with my readers.

Ini kambing korban tahun 1966. A week before D-Day, my late grandfather (whom I had the privilege of calling him "Ayah" instead of "Tok Ayah" like the rest of my cousins) gave me the green light and "license" to play with the ram before the actual "slaughtering day". I had a lot of fun horsing around on his back - I dunno about him though ...

Atas tangga depan, of our old house at AJ46 Kubang Kerian - with Mama, Ha and Na (clockwise).


In front of our family house - we had quite a big compound ... Mama is a "strong" woman - she single-handedly raised all of us to become "the human" we are today. She's both a mother and a father to the three of us. We love you very much Mama !

With my two lovely sisters - Saniha (Ha) & Salina (Na). Notice the close bonding (& binding ?) we have (and still do) among siblings - holding hands & everybody wearing the same selipar Jepun ... kami mmg rapat. Alhamdulillah - hasil didikan Mama tercinta.

KILLER IDEA vs IDEA KILLER

Imagine one morning while driving to work, you’re suddenly hit with a great idea. It could be just anything – may be a better way to disseminate info among staff, or a completely new marketing strategy for your college, or simply a cheaper, more interesting and faster way to do your daily routine work !

You rush to your workplace and coincidentally meet your boss at the punch-card machine counter. With full enthusiasm you immediately describe your great idea to him.

And without a second’s thought, he responded: “Oh, we already did that last year. It didn’t work.” And he just moves on, leaving you behind, speechless ...

Your killer idea has just met an idea killer !

WHAT IS AN IDEA KILLER ?

Idea killers are phrases or negative comments that people make without thinking when they first hear an idea. Here are some of the common ones:

• “That’s really stupid.”

• “Our college don’t operate like that.”
• “The bursar will never buy it.”
• “The CEO will hate it.”
• “Other college has already done that.”
• “Gimme a couple of weeks and I’ll get back to you.”
• “Let’s set up a committee to investigate its viability.”
• “May be in a couple of years from now.”
• “I don’t think it’ll work.”

GIVE IT A CHANCE TO GROW

Ideas, especially great ones, are fragile. Therefore they need openness, passion and nurturing to flourish.

So the next time someone comes up to you and shares with you his great idea, here is what you should do :

• Take 30 seconds to think why the idea would work
• Think of all the positive aspects of the idea
• Imagine how it could work
• Question your own assumptions
• Then, and only then, give him your feedback

You’ll be amazed at the difference this will make, especially to the person who initially brought up the idea. When you give an idea a moment’s thought, you give yourself the time to see and evaluate all its possibilities and the potential.

Of course, you can always critic the idea later on, but do it only after you have looked at all the positive aspects. This way, you may find "the seed" – that is the little germ of an idea that can blossom into great profits for your organization.

25 years KPTM - those were the days

Kolej Poly-Tech MARA (KPTM) celebrated her 25th anniversary on Nov 22nd 2008, at KPTM Cawangan Kuantan. One may ask why Kuantan ? I guess may be because it all started over there - at Maktab Sains MARA (MSM), Km.8 Jalan Gambang, Kuantan, Pahang Darul Makmur - way back in September 1983 ! At that time MSM was also popularly known as MARA Community College (MCC) - lebih glamor !!

I joined MCC Kuantan in October 1987 as a Computer Science Lecturer. Masa tu, jalan masuk ke kolej pun masih berlopak & berselut lagi ... The following are some of the old photos that I still keep to remind me of the good old days working at KPTM, especially Kuantan ...

Feb 1988 - my first Faculty Meeting. Everybody looked very "engrossed" listening to the President (now known as Pengarah). Cam tak muka2 "lama" kat gamba nih ? On my left is Pn Nurul (now Pengurus AS'AD in HQ - some may disagree w me but she's the best boss I've ever worked with - TQ Pn Nurul, may Allah bless u & ur family!), Zeek Mo (now w TM), Hj Ghozali (now Pengarah Bangi), En AbdAzmi (now TRA), Man Roi (now lecturer at UPSI - sedang menjalani ibadah Haji). Yang membelakangi kamera tu pulak Hj Mokhtar (now with YPM). In the foreground (berbaju putih) seated sebelah saya tu ialah arwah Wan Abd Azim (AlFatihah ...). His wife Kak Wan Ubai (also our ex-lecturer) is now a Senator in Kelantan. Syabas - Takbiiir !

April 1988 - Staff lunch at Academic Block Fasa II. Everybody were very close with each other. Admin-academics work and eat together-gether ... Life was very simple then ... even with the very basic facilities. Kerja seronok - we were all like brothers & sisters. Barisan depan tu HawaNazreen, Eshlaini (now in HQ), Salma & Wan Ubai (now a Senator).

The 1996 KYPM Annual Sports held at Trolak. I played tennis, partner w Rahman Art (squatting on the far right). He's an all-rounder person - asal orang Perak tapi dah bermastautin di Kuantan. Siapa tak kenal Rahman Art maknanya dia bukan orang Kuantan lah. Anyway, our Kuantan team came out Johan Keseluruhan for that year. Just look at the trophies & medals we've collected... tak muat bas !! Poyo abis ...

As KJ CS, sedang bagi hadiah perkahwinan Hjh Wan Asiah & her husband (yg pakai spec tu). Guess what, now she's my boss pulak (KJ CS) in KL. Camtu la hidup ni ... kejap kat atas, kejap kat bawah ... Yang handsome w moustache tu pulak ialah Mr Kamal (lecturer CS Kuantan).


Yang ni pulak, Farewell untuk Hj Ghozali Lamijan (fondly known as GL - bukan guarantee letter tau). Amboi amboi amboi ... lebatnya misai dia ! At that time GL was our Ketua Jabatan CS. But a week or so after that he changed his mind and came back to Kuantan ! Kuang kuang kuang ... On the left & right is Azmi MdNor (now in Bangi) and Khalijah (still in Kuantan) respectively.

One of the many Annual Hari Keluargas we had - this one was held at Taman Pertanian Kuantan (a very nice place for gatherings !). Notice Pn Rosmawati Hamzah dok pegang kotak beracun tuh ... masa tuh masih Cik lagi ! Yang bertudung labuh tu Hjh Noraziah (now w UUM) & the green T-shirt is Jamilah. Looking on is Rahman Akhir.


Barisan pensyarah elite ... muda trang tang tang ! Notice we used to have many kaum Adam at that time. But today majority of our lecturers r kaum Hawa. 5th from left (between Rahman & myself) is En Ismail Junuh, now the Pengarah of KPTM Kuantan. On my left was our English expert Ahmad Nor (baru pencen dgn UUM). Man Black & Ku Mat Akhir is 1st & 2nd from the right. Second from left besides Ust Romli is arwah Fauzi (AlFatihah ...)

After Commencement Ceremony kat Pusat Islam KL. Masa tuh kita ada pensyarah Mat Salleh Amerika - Mr Jan Sarna (on the far left). Next to him is Fakaruddin, Kamal Zamli, Asri, Hafiz Yatim (now a senior reporter w NST), Mr K (Kunjapan) - ex-senior teacher yang sangat berpengalaman orang nya (I wonder where he is now ...), Khairil Wahidin (now a Dr w UPM), Rahman Art (my best sparing partner ever) and of course yours truly.

As a young Ketua Jabatan CS. Look at the desk ... mmg messy dari dulu lagi ! ;)

Our MCC Kuantan Volleyball "Dream-ing" Team... Yang tak pakai jersey berdiri sebelah kiri tuh is my good friend, a Libero player - now Timbalan Rektor Akademik. Recognize him ? On his left is Man Black (now a Dr at UiTM). Next to him is Man Roi (now at UPSI). Squatting on far right (w dark glasses) is another good friend, Dr Joe Janggo (now a Dean at one College Univ in Melaka - apa nama tuh cek dah lupa ?). Both of us were "setters" for the team.


The "original" team of CS Dept at MCC Kuantan. GL, Azmi Md Noor, yours truly, Dr Suhaya, Khalijah and two programmers (sorry dah lupa nama). Noticed the "green screen" computers ? Masa tu kita pakai komputer Seiko & HP3000 mini-mainframes. Compilers have to be uploaded from large magnetic tapes on daily basis. Kalau ada PC pun kita pakai dengan floppies 5 1/4". At that time where got laptops & UMPC one ... ;) Thumbdrives, CDs & SD cards were never heard of. But we survived !


As KJ CS kena layan VIP (Dr Nik Safiah) melawat Computer Dept. Hj Fattah (our President at the time) is on the left. Cam tak lady yang pegang payung hitam tuh ?

Ini tengah makan malam kat chalet selepas penat ber-Treasure Hunt dari Kuantan ke Sekayu (Terengganu). Itu kat depan tengah pelahap ialah En Midon - warden paling popular kat Kuantan, esp among the girls ?? On my left is Dzulkarnain, Hissan & Rahman Hassan.

Taken on the hilltop of Bukit Panorama, Sg Lembing together with my good friend Woakil-uddin - our Math lecturer from Bangladesh. Saja bawak dia melawat tempat2 menarik di sekitar Kuantan ... Now he's attched w UNITEN & currently reading PhD in UK.


Gamba nih semasa CS Dept wat BBQ Picnic dengan CS Club (pelajar) kat Pantai Balok, Kuantan. Tu dok tengah cari player untuk acara Tarik Tali ... En Ghazali Bahari (yg tengah sapu mata dek ditiup angin Laut Cina Selatan) is standing next to me. Ingat lagi kat dia tak ? He was our Pendaftar at Kuantan. Now a senior MARA Officer.


In the office - as Timbalan Pengarah Pentadbiran & Hal Ehwal Pelajar. Those were the days ... the wonderful days working with KPTM... Kuantan ! (TQVM to all those original warga KPTM Kuantan. You've made my life beautiful and colorful ! - You're always close to my heart.)

In 2003 I was transferred to HQ/KL as Ketua Unit R&D. Then in 2005 transferred to Alor Star as Pengarah Kolej. In 2007 I requested to come back to KL.

Today, 21 years later, I'm still in the system, but now in KPTM Kuala Lumpur as Senior Lecturer. Working life & atmosphere is definitely "different" now ... too much politicking I guess ! Education now seems to be 2nd in the list ... :(

EPF CONTRIBUTIONS - To cut or not to cut ?

In light of the current economic slowdown, our government has come up with various strategies to increase rakyat’s extra disposable income and hoping that they will in turn spend that extra money to boost up our economic activity.

This is done by allowing employees to opt for a reduction of their contribution to the Employees Provident Fund (EPF) for two years from 1st January 2009 from 11% of salary to 8%.

No doubt that 3% may have good effects from the standpoint of the country, but a negative one on the long-term retirement savings of those affected, which for most of us EPF contributors is already too low.

Some of us would argue that it is just a small amount and therefore would not affect much of their retirement needs, but that is not the right way to look at it. It seems that many have failed to realize its implications in the long run.

From my personal standpoint, there is great need to put aside enough money for our retirement years. What more with the retirement age set so low at 55 and today’s longer life expectancy of at least 20 years beyond.

Every little bit helps. In fact, we should actually be looking at how to increase (rather than decrease) both the employers and employee contributions to the EPF so that our retirement savings becomes bigger.

We should therefore make every effort to resist (or rather reject ?) any change that would make our retirement benefit smaller.

I am also disappointed that the government has decided to use this strategy whereby the employees have to fill up a form to let the EPF know that we’re keeping our contribution at 11%. That means if we don’t do anything, may be due to forgetfulness or just couldn’t find the right time to do so, then EPF will assume that we have agreed to opt for the lower contribution of 8%.

To me, the better way to do this is the other way around. That is, to require those who want to change from the current 11% to 8% to fill up the form instead !

This clearly shows that the strategy used by the government is specially designed to ensure that most people will unwittingly agree to opt for the lower contribution by not doing anything !

In this aspect, I feel the government has neglected its duty to help ensure we have enough savings for our retirement.

No matter how much or how little we earn, it’s very crucial that we save some of it – at least 10% of our salary. For most of us, EPF is an enforced savings. Therefore, it must be left untouched or used only for the most important purpose in our life.

In this particular situation, I personally feel that we must be proactive and take matters into our own hands. So when EPF sends its forms out to our respective employers, make sure we immediately fill it up and state VERY CLEARLY that we want to keep our contribution at 11%.

The difference will mean thousands of ringgit many years down the line, money that will help us live better in our old age. InsyaAllah.