Item Response Theory

Item Response Theory (IRT) assumes that there is a correlation between the score gained by a candidate for one item/test (measurable) and their overall ability on the latent trait which underlies test performance (which we want to discover). Critically, the 'characteristics' of an item are said to be independent of the ability of the candidates who were sampled.

Item Response Theory comes in three forms: IRT1, IRT2, and IRT3 reflecting the number of parameters considered in each case.

For IRT1, only the difficulty of an item is considered,
(difficulty is the level of ability required to be more likely to correctly answer the question than answer it wrongly).
For IRT2, difficulty and discrimination are considered,
(discrimination is how well the question is at separating out candidates of similar abilities).
For IRT3, difficulty, discrimination and chance are considered,
(chance is the random factor which enhances a candidates probability of success through guessing.

IRT can be used to create a unique plot for each item (the Item Characteristic Curve - ICC). The ICC is a plot of Probability that the Item will be answered correctly against Ability. The shape of the ICC reflects the influence of the three factors:

Increasing the difficulty of an item causes the curve to shift right - as candidates need to be more able to have the same chance of passing.
Increasing the discrimination of an item causes the gradient of the curve to increase. Candidates below a given ability are less likely to answer correctly, whilst candidates above a given ability are more likely to answer correctly.
Increasing the chance raises the baseline of the curve.

This simple simulation allows the user to investigate the factors governing the shape of the Item Characteristic Curve. All three well known IRT models are represented (referred to as IRT1, IRT2 and IRT3) and Item Characteristic Curves can be super-imposed on one another to see how they relate.

Click to download this Item Response Theory interface as an IMS Content Package: itemPIF.zip. The CP may be opened in Managed Learning Environment software such as WebCT or in the Reload Tools.