What Is Item Difficulty? The Quiz Quality Metric Most Teachers Ignore

At a glance
The share who got it right, and why it matters
- 1
What p means
p is the proportion answering correctly, so a higher value means an easier item.
- 2
Aim for the middle
Average item difficulty of 0.5-0.7 differentiates students best.
- 3
Pair it with discrimination
A p of 0.50 could be a great item, a coin flip or a broken question.
- 4
Review the extremes
Items above 0.95 or below 0.15 are usually the ones to rewrite.
- 1.What the values mean
- 2.How to use it
- 3.Item difficulty + item discrimination
- 4.A worked example with real numbers
- 5.Guessing changes what counts as hard
- 6.Small classes: how far to trust p
- 7.How modern quiz tools surface this
- 8.Common mistakes
- 9.Item difficulty across high-stakes exams
- 10.A diagnostic protocol after a quiz
- 11.Why item difficulty matters less than discrimination
- 12.Frequently Asked Questions
- 13.Related reading
Short answer. Item difficulty (denoted *p*) is the proportion of students who answered a given quiz question correctly. A question that 80% of students got right has *p* = 0.80. The metric is poorly named — *higher* difficulty value = *easier* question — but it's a foundational stat for evaluating quiz quality.
What the values mean
- p = 1.0: Every student got it right. Either trivially easy, or a freebie. Not useful for distinguishing students.
- p = 0.80-0.90: Easy. Good for confidence-building, foundational checks.
- p = 0.50-0.70: Medium. Most useful for assessment — discriminates between students who know and don't know.
- p = 0.20-0.40: Hard. Good for stretch questions; use sparingly.
- p < 0.20: Either too hard for the cohort, or a broken question.
- p = 0: No one got it right. Almost certainly a broken question.
How to use it
For a typical quiz:
- Aim for an average difficulty of 0.5-0.7 across all items
- Don't have all easy items (p > 0.85) — the quiz doesn't differentiate
- Don't have all hard items (p < 0.4) — the quiz frustrates without measuring
- Mix difficulty levels so the quiz has a discrimination curve
Item difficulty + item discrimination
Item difficulty alone is incomplete. A question with p = 0.50 might be:
- A great question (top half of class gets it right, bottom half doesn't) — high item discrimination
- A coin-flip question (every student has 50/50 odds) — low item discrimination
- A *reverse-discrimination* broken question (bottom half gets it right, top half misses) — negative discrimination
Use both metrics together. A good quiz has medium-difficulty items with positive discrimination.
A worked example with real numbers
Take a 5-question quiz answered by 30 students. For each question, count the correct answers and divide by 30.
| Question | Correct (of 30) | p | Reading |
|---|---|---|---|
| Q1 | 29 | 0.97 | Near-freebie. Fine as an opener, tells you little |
| Q2 | 24 | 0.80 | Easy |
| Q3 | 18 | 0.60 | Medium, the most informative range |
| Q4 | 12 | 0.40 | Hard |
| Q5 | 4 | 0.13 | Review before you trust it |
The average difficulty is (0.97 + 0.80 + 0.60 + 0.40 + 0.13) / 5 = 0.58. That number is also the class average on the quiz: 87 correct answers out of 150 possible is 58%. Average item difficulty and mean score are the same figure seen from two sides, so if you want a higher or lower class average next time, you change it by swapping individual items, and the table tells you which ones.
Q5 deserves a closer look. If it is a four-option multiple-choice question, blind guessing alone would put about 7 or 8 of 30 students on the right answer. Only 4 got it right, which is below chance. That does not happen when students simply do not know; it happens when something pulls them towards a wrong option, either a wrong answer key or a strong misconception. Check the key first, then look at which option most students picked. If one wrong option took most of the votes, the fix is in the distractor design or in the teaching, not in the students.
Guessing changes what counts as hard
The lowest p you can expect depends on the question format, because students who know nothing still guess. A common rule of thumb in test design is to aim for a difficulty about halfway between the chance score and 1.0.
| Format | Chance score | Halfway target |
|---|---|---|
| True/false | 0.50 | 0.75 |
| Multiple choice, 3 options | 0.33 | 0.67 |
| Multiple choice, 4 options | 0.25 | 0.63 |
| Multiple choice, 5 options | 0.20 | 0.60 |
| Short answer or fill in the blank | Close to 0 | 0.50 |
So the same p means different things. A true/false item at p = 0.60 is barely above coin-flipping, while a short-answer item at p = 0.60 shows that most of the class could produce the answer unaided. Compare items of the same format, or adjust your expectations before comparing across formats.
Small classes: how far to trust p
In a class of 20, each student moves p by 0.05. Two students who misread the question turn a 0.70 item into a 0.60 item. With one small class, treat p as a rough reading:
- Act on the extremes (above 0.95, below 0.20). Those rarely flip with a few more students.
- Do not rank middle items against each other. A 0.55 and a 0.65 from 20 students are not reliably different.
- Reuse the question with another section or next year's class and pool the results. Sixty responses give a far steadier value than twenty.
How modern quiz tools surface this
Most LMSs and quiz platforms show item difficulty after each quiz administration:
- Canvas, Blackboard, Moodle: quiz statistics page
- SimpleQuizMaker: per-question analytics across all submissions
After every quiz, scan the difficulty distribution. The items with p > 0.95 (everyone right) and p < 0.15 (almost no one right) are usually the ones to review or rewrite.
Common mistakes
- Confusing the name. Higher value = easier. Not intuitive. Many teachers reverse this in their heads.
- Ignoring difficulty data when grading. The "everyone missed Q5" pattern usually means Q5 is broken, not that everyone failed to learn.
- Treating low p as a teaching problem when it's a question problem. Verify the question is well-written before concluding students don't know the material.
Item difficulty across high-stakes exams
For comparison, the average item difficulty on professional exams:
- AP exams: target around p = 0.60 — calibrated to differentiate scores 1-5.
- SAT: average p around 0.65; harder items in the back of each section.
- MCAT, GRE: adaptive, so item difficulty is selected per student to maintain ~50% expected correct rate.
- Bar exam, USMLE: average p around 0.55 — high-stakes filtering exams.
Classroom unit tests typically run easier (p around 0.75) because the goal is documenting that most students learned the material, not differentiating among them.
A diagnostic protocol after a quiz
After the first 10-20 students take a new quiz:
- Sort items by p. Anything with p > 0.95 or p < 0.20 gets reviewed.
- For high-p items: was this a freebie? If yes, fine. If you intended it to be hard, the wording probably gave it away.
- For low-p items: read the question aloud, work it yourself. If you struggle or notice ambiguity, the item is broken. If it's truly hard and well-written, keep it but note the difficulty.
- Check the discrimination index on borderline-difficulty items (p around 0.40-0.60). High discrimination = useful item; low or negative = revise.
After 5-10 administrations, you have a refined item bank with known difficulty. AP/MCAT/NCLEX item banks went through this process for years before reaching production.
Why item difficulty matters less than discrimination
Two questions with the same difficulty (p = 0.60) can have very different value:
- One is a "good 60%" — top students mostly right, bottom students mostly wrong (high discrimination)
- One is a "noise 60%" — random students right (low or zero discrimination)
The first is a productive item; the second is a coin flip dressed up as a quiz question. Always check both metrics; never use difficulty alone.
Frequently Asked Questions
Is a higher item difficulty value harder or easier?
Easier. Item difficulty is the proportion of students who answered correctly, so p = 0.90 is an easy question and p = 0.20 is a hard one.
How do I calculate item difficulty by hand?
Divide the number of students who answered the question correctly by the number who attempted it. If 18 of 30 students got it right, p = 18 / 30 = 0.60.
What is a good item difficulty for a multiple-choice question?
For most classroom quizzes, somewhere between 0.50 and 0.80 for the bulk of the items. A four-option question has a chance score of 0.25, so the halfway rule puts the target near 0.63.
Should I delete a question that everyone got right?
Not automatically. A question at p = 1.0 does not separate students, but it can still confirm that an essential point was learned or settle nerves at the start of a quiz. Remove it only if you meant it to be challenging.
Related reading
- What Is Item Discrimination?
- How to Write Good Quiz Questions
- How to Write Hard Quiz Questions
- Quiz Analytics — Teacher Guide
- What Is a Distractor (in Quiz Design)?
- Quiz and assessment glossary
- What Is Bloom's 2 Sigma Problem? The Tutoring-Effectiveness Puzzle
- What Is Desirable Difficulty? When Harder Is Better for Learning
Get weekly study & quiz tips
Join teachers and students who get practical tips on quizzing, active recall, and AI-powered learning.
Questions or a quiz of your own on this? Discuss this on r/SimpleQuizMaker
SimpleQuizMaker Editorial Team
Editorial team
How we create and review guides →
Practice with AI-generated quizzes
Try it on your own material
SimpleQuizMaker turns what you are already teaching or studying into practice questions. The free plan includes 5 quiz generations a month and needs no card.
Related posts
GlossaryWhat Is Item Discrimination? A Quiz Quality Metric
4 min
GlossaryWhat Is Retrieval Practice? The Evidence-Based Study Habit
3 min
GlossaryWhat Is Desirable Difficulty? When Harder Is Better for Learning
4 min
GlossaryWhat Is Bloom's 2 Sigma Problem? The Tutoring-Effectiveness Puzzle
4 min
GlossaryWhat Is a Distractor in Multiple Choice Questions?
4 min
GlossaryWhat Is the Testing Effect? The Research Behind Active Recall
4 min