SyberWorks e-Learning Podcast Series
SyberWorks e-Learning Podcast Transcript #18 Tips for Creating Online Tests (Transcript)
[music]
Announcer: SyberWorks Podcast. Learn any time, any place.
Mary Kay Lofurno: Welcome to the next edition of the SyberWorks e-Learning Podcast series. My name is Mary Kay Lofurno, and I'm the Marketing Director here at SyberWorks, and your host today.
SyberWorks specializes in custom e-learning solutions, learning management systems, and custom e-learning development for corporations, governments and non-profit institutions. In this edition, we are talking with Dana Fine, Senior Instructional Designer here at SyberWorks. She recently wrote an article on how to create online tests, and I thought we could get some good tips from her on the subject.
[music]
Mary Kay: Hi, Dana, it's great to see you. It's been a while since we've had you here on the SyberWorks s-Learning Podcast series. So, how have you been?
Dana Fine: Great. How are you?
Mary Kay: Good. All right, let's get to it. Why did you write the article on how to write online test questions? It seems to me that if you're designing a course, the part where you create the test would not be that hard to figure out.
Dana: Writing good online test questions can be a lot harder than you would think.
Mary Kay: OK, Dana, how come?
Dana: That test writer has to determine, not only in the course content, how the objectives of the course will be filled through online learning, but -- and this is a big but -- how to test how well those objectives were grasped by the student. The test need to test how well they retain the material, the major points of that material, and most importantly, it should be a good test.
That begs the question, what makes a good test? A good test should have the correct level of difficulty for the material. Each question should be clear and have clear-cut answers. In addition, a student should be able to score 100% on a good online test. A good online test will test students on that material that they need to know, and will have clear-cut questions and answers of varying types, that is single-correct, multiply-correct, true-false, maybe some fill-in-the-blanks. And will have a significant number of questions so that the test can draw from a question bank, a bank of say, 100 or so questions to make different tests for every student, to avoid those students who would shoot themselves in the foot by cheating.
Mary Kay: OK, that sounds good. Let's talk about some of your tips for writing great online tests.
Dana: As I wrote in the article, there are several points to keep in mind as you write your tests. The most important tip to keep in mind is the main objectives of the course. You have to review your course outline from lesson to topic to sub-topic, and determine what each sub-topic covered. Does the student need to retain those main ideas? Does that student need that specific knowledge, a specific term or procedure? Do you need to test that the student picked up that idea?
Once you have determined what you are testing for, your must determine how hard the test will be. Online tests, in my opinion, should not be terribly difficult. Instructors in the offline world will sometimes pride themselves on having tests that no one can get 100% on. The online world is radically different.
Online tests for that specific knowledge that the student wants to acquire, be it just-in-time training, skills training for certification, or any other type of training. Online tests, like offline tests, should have some questions that are very easy, so that the student has some confidence in indicators of what the test will cover, such as true-false questions or the more obviously written single-correct questions. Veering into more difficult questions of any type can provide the build-up to the expectations of achievement.
The online world, unlike the offline world, allows a student to take a test more than once. Determining how many times the student should be able to take the test is usually based on the purpose of the course and the exam.
Taking a test several times, as long as there is a good question bank to draw upon, will improve the retention of the material by the student.
One nice thing about the online world, students are not embarrassed by taking a test over and over. No one is looking over their shoulders, no one will remark on how long it took them to pass. And if the student wants to review the material by taking the exam again, that's all to the good.
The question bank may also determine how many times a student may take an online test. If you have a 20-question test and only 40 questions in your bank, you may want to limit the number of times a student can take the test.
But be reasonable; in my experience, three tries are the minimum number of tries, even if your question bank is not large. However, warn the student prior to the exam, otherwise, complaints will mount.
Mary Kay: Hmm. Well, that makes a lot of sense. How does someone know whether or not they've created a good online test, Dana?
Dana: A good online test tests the material that the course writer wanted the student to retain after taking the course. A good online test has terminology and grammar that the students understand after taking the course. If they don't understand the terminology, for example, the exam -- and even the course -- needs to be reviewed.
Make sure that there is one, and only one, way to interpret each question. This is more difficult for the course writer than it appears. Your answers to a question, for example a single-correct question, may make the question seem to indicate a different meaning. The question and its answers need to be in synch, and the wrong answers need to be plausible.
I use caution in designing questions. Simple guidelines for designing questions include: that true-false questions are best used to instill confidence in the student. They should not be used to undermine confidence as a trick question. Online test should really not have trick questions. Not only is there no instructor to explain these tricks, or even an instructor online to explain the reasoning via discussion groups or other online interactions.
Single-correct questions should, at maximum, have four possible answers, of which only one is correct. Multiply-correct question should also have only four possible answers maximally, of which as least one should be incorrect.
Online students are not shy in their feedback. The anonymity of being online insures that the feedback will really let you know what your students think.
You will know how the students felt about an exam in detail, if they feel that it's poorly written and unfair. In addition, question analysis can assist you in determining whether the test questions were good questions. If every student that took the test got a specific question wrong, that question is not a good question, no matter how it seems to the test writer.
Mary Kay: Dana, this is all great information. Thanks for joining us today. Dana's article can be found in the media center at the SyberWorks website.
Dana: Thank, Mary Kay, great chatting with you about my favorite topic.
Mary Kay: As always, Dana, it's great to see you.
Thank for listening to our podcast today on tips for writing online tests, with Dana Fine, Senior Instructional Designer here at SyberWorks, on our e-Learning Podcast series. Have a great day.
[music]
Announcer: SyberWorks Podcast. Learn anytime, anyplace.