Skip to content
Glossary

What Is Item Difficulty? The Quiz Quality Metric Most Teachers Ignore

A gauge card, a teal dial, a magnifying glass, a pencil and a stack of cards

At a glance

The share who got it right, and why it matters

  1. 1

    What p means

    p is the proportion answering correctly, so a higher value means an easier item.

  2. 2

    Aim for the middle

    Average item difficulty of 0.5-0.7 differentiates students best.

  3. 3

    Pair it with discrimination

    A p of 0.50 could be a great item, a coin flip or a broken question.

  4. 4

    Review the extremes

    Items above 0.95 or below 0.15 are usually the ones to rewrite.

Takeaway: When everyone misses a question, suspect the question first and check its discrimination.
Share:XLinkedIn

Short answer. Item difficulty (denoted *p*) is the proportion of students who answered a given quiz question correctly. A question that 80% of students got right has *p* = 0.80. The metric is poorly named — *higher* difficulty value = *easier* question — but it's a foundational stat for evaluating quiz quality.

What the values mean

  • p = 1.0: Every student got it right. Either trivially easy, or a freebie. Not useful for distinguishing students.
  • p = 0.80-0.90: Easy. Good for confidence-building, foundational checks.
  • p = 0.50-0.70: Medium. Most useful for assessment — discriminates between students who know and don't know.
  • p = 0.20-0.40: Hard. Good for stretch questions; use sparingly.
  • p < 0.20: Either too hard for the cohort, or a broken question.
  • p = 0: No one got it right. Almost certainly a broken question.

How to use it

For a typical quiz:

  • Aim for an average difficulty of 0.5-0.7 across all items
  • Don't have all easy items (p > 0.85) — the quiz doesn't differentiate
  • Don't have all hard items (p < 0.4) — the quiz frustrates without measuring
  • Mix difficulty levels so the quiz has a discrimination curve

Item difficulty + item discrimination

Item difficulty alone is incomplete. A question with p = 0.50 might be:

  • A great question (top half of class gets it right, bottom half doesn't) — high item discrimination
  • A coin-flip question (every student has 50/50 odds) — low item discrimination
  • A *reverse-discrimination* broken question (bottom half gets it right, top half misses) — negative discrimination

Use both metrics together. A good quiz has medium-difficulty items with positive discrimination.

A worked example with real numbers

Take a 5-question quiz answered by 30 students. For each question, count the correct answers and divide by 30.

QuestionCorrect (of 30)pReading
Q1290.97Near-freebie. Fine as an opener, tells you little
Q2240.80Easy
Q3180.60Medium, the most informative range
Q4120.40Hard
Q540.13Review before you trust it

The average difficulty is (0.97 + 0.80 + 0.60 + 0.40 + 0.13) / 5 = 0.58. That number is also the class average on the quiz: 87 correct answers out of 150 possible is 58%. Average item difficulty and mean score are the same figure seen from two sides, so if you want a higher or lower class average next time, you change it by swapping individual items, and the table tells you which ones.

Q5 deserves a closer look. If it is a four-option multiple-choice question, blind guessing alone would put about 7 or 8 of 30 students on the right answer. Only 4 got it right, which is below chance. That does not happen when students simply do not know; it happens when something pulls them towards a wrong option, either a wrong answer key or a strong misconception. Check the key first, then look at which option most students picked. If one wrong option took most of the votes, the fix is in the distractor design or in the teaching, not in the students.

Guessing changes what counts as hard

The lowest p you can expect depends on the question format, because students who know nothing still guess. A common rule of thumb in test design is to aim for a difficulty about halfway between the chance score and 1.0.

FormatChance scoreHalfway target
True/false0.500.75
Multiple choice, 3 options0.330.67
Multiple choice, 4 options0.250.63
Multiple choice, 5 options0.200.60
Short answer or fill in the blankClose to 00.50

So the same p means different things. A true/false item at p = 0.60 is barely above coin-flipping, while a short-answer item at p = 0.60 shows that most of the class could produce the answer unaided. Compare items of the same format, or adjust your expectations before comparing across formats.

Small classes: how far to trust p

In a class of 20, each student moves p by 0.05. Two students who misread the question turn a 0.70 item into a 0.60 item. With one small class, treat p as a rough reading:

  • Act on the extremes (above 0.95, below 0.20). Those rarely flip with a few more students.
  • Do not rank middle items against each other. A 0.55 and a 0.65 from 20 students are not reliably different.
  • Reuse the question with another section or next year's class and pool the results. Sixty responses give a far steadier value than twenty.

How modern quiz tools surface this

Most LMSs and quiz platforms show item difficulty after each quiz administration:

  • Canvas, Blackboard, Moodle: quiz statistics page
  • SimpleQuizMaker: per-question analytics across all submissions

After every quiz, scan the difficulty distribution. The items with p > 0.95 (everyone right) and p < 0.15 (almost no one right) are usually the ones to review or rewrite.

Common mistakes

  • Confusing the name. Higher value = easier. Not intuitive. Many teachers reverse this in their heads.
  • Ignoring difficulty data when grading. The "everyone missed Q5" pattern usually means Q5 is broken, not that everyone failed to learn.
  • Treating low p as a teaching problem when it's a question problem. Verify the question is well-written before concluding students don't know the material.

Item difficulty across high-stakes exams

For comparison, the average item difficulty on professional exams:

  • AP exams: target around p = 0.60 — calibrated to differentiate scores 1-5.
  • SAT: average p around 0.65; harder items in the back of each section.
  • MCAT, GRE: adaptive, so item difficulty is selected per student to maintain ~50% expected correct rate.
  • Bar exam, USMLE: average p around 0.55 — high-stakes filtering exams.

Classroom unit tests typically run easier (p around 0.75) because the goal is documenting that most students learned the material, not differentiating among them.

A diagnostic protocol after a quiz

After the first 10-20 students take a new quiz:

  1. Sort items by p. Anything with p > 0.95 or p < 0.20 gets reviewed.
  2. For high-p items: was this a freebie? If yes, fine. If you intended it to be hard, the wording probably gave it away.
  3. For low-p items: read the question aloud, work it yourself. If you struggle or notice ambiguity, the item is broken. If it's truly hard and well-written, keep it but note the difficulty.
  4. Check the discrimination index on borderline-difficulty items (p around 0.40-0.60). High discrimination = useful item; low or negative = revise.

After 5-10 administrations, you have a refined item bank with known difficulty. AP/MCAT/NCLEX item banks went through this process for years before reaching production.

Why item difficulty matters less than discrimination

Two questions with the same difficulty (p = 0.60) can have very different value:

  • One is a "good 60%" — top students mostly right, bottom students mostly wrong (high discrimination)
  • One is a "noise 60%" — random students right (low or zero discrimination)

The first is a productive item; the second is a coin flip dressed up as a quiz question. Always check both metrics; never use difficulty alone.

Frequently Asked Questions

Is a higher item difficulty value harder or easier?

Easier. Item difficulty is the proportion of students who answered correctly, so p = 0.90 is an easy question and p = 0.20 is a hard one.

How do I calculate item difficulty by hand?

Divide the number of students who answered the question correctly by the number who attempted it. If 18 of 30 students got it right, p = 18 / 30 = 0.60.

What is a good item difficulty for a multiple-choice question?

For most classroom quizzes, somewhere between 0.50 and 0.80 for the bulk of the items. A four-option question has a chance score of 0.25, so the halfway rule puts the target near 0.63.

Should I delete a question that everyone got right?

Not automatically. A question at p = 1.0 does not separate students, but it can still confirm that an essential point was learned or settle nerves at the start of a quiz. Remove it only if you meant it to be challenging.

Try it yourself, free →

Get weekly study & quiz tips

Join teachers and students who get practical tips on quizzing, active recall, and AI-powered learning.

Share:XLinkedIn

Questions or a quiz of your own on this? Discuss this on r/SimpleQuizMaker

SimpleQuizMaker Editorial Team

Editorial team

How we create and review guides →

Practice with AI-generated quizzes

Try it

Quiz analytics →

See which questions a group missed and which wrong answer pulled them.

Try it on your own material

SimpleQuizMaker turns what you are already teaching or studying into practice questions. The free plan includes 5 quiz generations a month and needs no card.

Sign up free — no card required