5 Tips for Examining Testwiseness & Question Flaws in USMLE [The Writer’s Series]

As you prepare for your upcoming USMLE exam, it’s helpful to go “behind the scenes” to see how the NBME instructs their writers to write question items. Having insight into this process will help you better approach questions on your actual exam.

A good question must satisfy two basic criteria. First, the question item must address important and relevant content. Second, the question must be well-structured. It’s nearly impossible to assess one’s proficiency on a high-yield topic if the question item is not well-structured.

Technical item flaws are what cause questions to be considered as not well-structured. These flaws are split into two buckets: issues related to testwiseness and issues related to irrelevant difficulty.

Testwiseness is defined as “any skill which allows a student to choose the correct answer on an item without knowing the correct answer.”

Test takers with high levels of testwiseness are often able to determine the correct answer by using unintentional clues that are in the question. Let’s take a look some of these technical item flaws that can help you become more testwise.

BECOME MORE TESTWISE FOR THE USMLE

1. GRAMMATICAL CUES

One technical flaw is when one or more distractors don’t follow grammatically from the stem. In the example below, the term “intravenous administration of” should trigger the elimination of choices A and C, leaving just three choices to narrow down from.

Q: A 60-year-old man is brought to the emergency department by the police, who found him lying unconscious on the sidewalk. After ascertaining that the airway is open, the first step in management should be intravenous administration of:

A. examination of cerebrospinal fluid
B. glucose with vitamin B1 (thiamine)
C. CT scan of the head
D. phenytoin
E. diazepam

2. LOGICAL CUES

Next, look to see if a subset of the options are collectively exhaustive. In the example below, choices A, B and C collectively cover all options (poor, middle class, rich and all), so that effectively eliminates D and E right away.

Q: Crime is

A. equally distributed among the social classes
B. overrepresented among the poor
C. overrepresented among the middle class and rich
D. primarily an indication of psychosexual maladjustment
E. reaching a plateau of tolerability for the nation

3. ABSOLUTE TERMS

Look out for when terms such as “always” or “never” are used in options. In the next example, a testwise student would eliminate choices C and D based on the fact that both use the absolute term “never.” It’s less likely a choice containing absolute terms would be true versus one that is worded less absolutely.

Q: In patients with advanced dementia, Alzheimer’s type, the memory defect

A. can be treated adequately with phosphatidylcholine (lecithin)
B. could be a sequela of early parkinsonism
C. is never seen in patients with neurofibrillary tangles at autopsy
D. is never severe
E. possibly involves the cholinergic system

4. REPETITIVE WORDS 

Be aware of when a word or phrase is included in the stem and in the correct answer. In the example below, the word “unreal” is mentioned. Subsequently, “derealization” is answer choice C, which would result in a testwise student choosing it as the correct answer. Also be on the lookout for veiled, but similar phrases like “cardio” and “heart palpitations.”

Q: A 58-year-old man with a history of heavy alcohol use and previous psychiatric hospitalization is confused and agitated. He speaks of experiencing the world as unreal. This symptom is called

A. depersonalization
B. derailment
C. derealization
D. focal memory deficit
E. signal anxiety

5. CONVERGENCE STRATEGY 

The correct answer includes the most elements in common with the other options. This strategy analyzes the words used in question choices.

The theory is that the question writer will mention correct elements more than incorrect ones. Below we can see that ‘anionic’ is mentioned once, and ‘cationic’ and ‘uncharged form’ are mentioned twice.

Moving to the second element of the answer choices, we see that “acting from inside the nerve membrane” is mentioned three times whereas outside the membrane is only mentioned twice. A testwise student would look at these combinations and choose between options B and D.

Q: Local anesthetics are most effective in the

A. anionic form, acting from inside the nerve membrane
B. cationic form, acting from inside the nerve membrane
C. cationic form, acting from outside the nerve membrane
D. uncharged form, acting from inside the nerve membrane
E. uncharged form, acting from outside the nerve membrane

Now that you’ve become aware of all these signals, test yourself on these USMLE Step 1 practice questions.