Guidelines for Constructing Multiple Choice Items1) In composing the multiple choice item stems, use terms whose definitions are likely to be precise in minds of examinees. Avoid unnecessarily unfamiliar terminology. The difficulty of an item should evolve from the subject matter rather than from the wording. Example: After firing, an avalanche control cannon recoils at 20 feet per second. What would be the recoil speed of another cannon, which is twice as heavy and fires the same projectile? A student completely unfamiliar with the term "recoil" may face unintended extra difficulty. 2) Avoid vague qualitative modifiers such as many, large, much, small, old, and important. Example: Much of the process of food absorption takes place in the:
The qualifier "much" provides a source of unnecessary vagueness likely to confuse or mislead. A more specific qualifier such as "most" would have allowed a more precise interpretation of the item’s intent. 3) Avoid complex or awkward word arrangements. Also, avoid use of negatives in the stem as this may add unnecessary comprehension difficulties. Example: (Better) 4) Avoid double negatives. This is another instance of choosing a simpler sentence construction so the item performance is less dependent upon grammatical skills.
Example: (Better) 5) Avoid trick statements with some minor misleading word or spelling anomaly. An experienced exam taker may profit from his detection of such a ploy, despite a lack of knowledge in the subject matter. Example: True or False. The Kiuang is an important means of access to the interior of China. The incomplete statement of the name of the river, which actually is an important access route to inland China, presents an unreasonable attempt to mislead or trick the examinee. 6) Avoid the exact wording of a source material. This places too much emphasis on simple memory and barely tests cognitive acquisition at the knowledge level. 7) Avoid specific determiners or give-away qualifiers. Students quickly learn that strongly worded statements are more likely to be false than true, for example, statements with "never" "no" "all" or "always." Moderately worded statements are more likely to be true than false. Statements with "many" "often" "sometimes" "generally" "frequently" or "some" should be avoided. 8) With true or false questions, avoid a grossly disproportionate number of either true or false statements. It may be best to have slightly more false since examinees guessing at a true or false statement are more likely to choose true, although many test experts insist that all patterns should be avoided. 9) Each item stem should be as short as possible; otherwise you risk testing more for reading and comprehension skills. 10) Distracters should be equally plausible and attractive. Example: The English
based their claim to territorial rights in North America on the explorations
of: a. Henry Hudson b. Ferdinand Magellan c. Sir Walter Raleigh d. Americus Vespucias e. Hernando De Soto
11) All multiple choice options should be grammatically consistent with the stem. Example: The discovery by Oersted of the magnetic effect of an electric current led to development of the: a. Fluorescent light b. Electric motor c. X-rays d. Tungsten Filament bulb e. Electrolytic
capacitor The utility of choice C as a distracter is greatly diminished. Item writers often tend to devote more energy to the precision of phrasing of the correct response. 12) The length, explicitness, or degree of technicality of alternatives should not vary with correctness. Example: In the Mediterranean
region, cattle raising is handicapped by: a. A hot climate b. Lack of demand for beef c. A lack of sufficient productive pasturage d. High labor costs
13) Avoid stems that reveal the answer to another item. 14) Avoid alternatives that are synonymous with, include or overlap others. Example: What makes the diaphragm of a b. Changes in temperature c. Chemical changes d. Variations in pressure e. Variations in heat
15) Avoid presenting sequenced items in the same order as in the text. Example: True or False. Monsoon Asia includes India, Eastern China, and Japan. The statement is true. If the item writer extracted the series of countries in the order they appear in the text, a correct response becomes more dependent on recognition of the sequence sounds rather than knowledge of the sequence components. 16) Avoid use of assumed qualifiers that many examinees may not be aware of. Example: The principal
difference in character between the Northern and Southern Colonies was
due, primarily, to difference in their: a. Language and Literature b. Topography and climate
17) Avoid use of unnecessary words or phrases, which are not relevant to the problem at hand (unless such discriminating ability is the primary intent of the evaluation). The item’s value is particularly damaged if the unnecessary material is designed to distract or mislead. This once again, gives an inordinately strong influence to reading comprehension as a determinate of performance. Example: The radius
of a circle is one half of the diameter. The circumference of a circle,
with a diameter of 1.73 inches, is about: a. 2.6 b. 3.5 c. 5.5 d. 11.0 e. 16.0 The first sentence is really quite irrelevant to a solution of the problem
and lowers the value of the question as strictly a determinate of knowledge
of calculation of circumference. 18) Avoid use of non-relevant sources of difficulty such as requiring a complex calculation when only knowledge of a principle is being tested. Note in the previous example, a given whole number diameter would have allowed just as accurate an indication of the examinees ability to calculate circumference. The use of a decimal fraction changes the level of difficulty. 19) Avoid extreme specificity requirements in responses. Example: In the 1932 presidential election, Franklin Roosevelt
received how many electoral votes. 20) Include as much of the item as possible in the stem. This allows less repetition and shorter choice options. 21) Use the "None of the above" option only when the keyed answer is totally correct. When choice of the "best" response is intended, "none of the above" is not appropriate, since the implication has already been made that the correct response may be partially inaccurate. 22) Note that use of "all of the above" may allow credit for partial knowledge. In a multiple option item, (allowing only one option choice) if a student only knew that two (2) options were correct, he could then deduce the correctness of "all of the above". This assumes you are allowed only one correct choice. 23) Having compound response choices may purposefully increase difficulty of an item. Example: Businessmen are central figures in which of the following pairs of American novels: a. Arrowsmith and As I Lay Dying b. A Farewell to Arms and Washington Square c. Babbitt and The Rise of Silas Lapham
Note: This approach is most effective when each option contains at least one correct component. 24) The difficulty of a multiple choice item may be controlled by varying the homogeneity or degree of similarity of responses. The more homogeneous, the more difficult the item. Example: Thailand is located in: a. Southeast Asia b. Eastern Europe c. South America d. East Africa e. Central America
Thailand
is located next to: a. Laos and Kampuchea b. India and China c. China and Malaya d. Laos and China e. India and Malaya |