Adaptive Quiz: CAT (Computer-Adaptive Testing) implementation for Moodle

Create tests that efficiently measure users’ abilities by adapting the questions difficulty to the estimation of user’s ability.
The Adaptive Quiz activity enables a teacher to create tests that efficiently measure the takers’ abilities. Adaptive tests are comprised of questions selected from the question bank that are tagged with a score of their difficulty. The questions are chosen to match the estimated ability level of the current test-taker. If the test-taker succeeds on a question, a more challenging question is presented next. If the test-taker answers a question incorrectly, a less-challenging question is presented next. This technique will develop into a sequence of questions converging on the test-taker’s effective ability level. The test stops when the test-taker’s ability is determined to the required accuracy.
The Adaptive Quiz activity uses the “Practical Adaptive Testing CAT Algorithm” by B.D. Wright published in Rasch Measurement Transactions, 1988, 2:2 p.24 and discussed in John Linacre’s “Computer-Adaptive Testing: A Methodology Whose Time Has Come.” MESA Memorandum No. 69 (2000) .
This Moodle activity module was created as a collaborative effort between Middlebury College and Remote Learner . Later on it was adopted by  Vitaly Potenko  to keep it compatible with new Moodle versions and enhance with new features.
Below you’ll find short documentation on the plugin to explain its essential concepts and flows.
The Question Bank
To begin with, questions to be used with this activity are added or imported into  Moodle’s question bank. Only questions that can automatically be graded may be used.  As well, questions should not award partial credit. The questions can be placed in  one or more categories.
This activity is best suited to determining an ability measure along a unidimensional  scale. While the scale can be very broad, the questions must all provide a measure of  ability or aptitude on the same scale. In a placement test for example, questions low  on the scale that novices are able to answer correctly should also be answerable by  experts, while questions higher on the scale should only be answerable by experts or  a lucky guess. Questions that do not discriminate between takers of different  abilities on will make the test ineffective and may provide inconclusive results.
Take for example a language placement test. Low-difficulty vocabulary and  reading-comprehension questions would likely be answerable by all but the most novice  test-takers. Likewise, high-difficulty questions involving advanced grammatical  constructs and nuanced reading-comprehension would be likely only be correctly  answered by advanced, high-level test-takers. Such questions would all be good  candidates for usage in an Adaptive Test. In contrast, a question like “Is 25¥ a good  price for a sandwich?” would not measure language ability but rather local knowledge  and would be as likely to be answered correctly by a novice speaker who has recently  been to China as it would be answered incorrectly by an advanced speaker who comes  from Taiwan — where a different currency is used. Such questions should not be  included in the question-pool.
Questions must be tagged tagged with a ‘difficulty score’ using the format  ‘adpq_ n ‘ where n is a positive integer, e.g. ‘adpq_1’ or ‘adpq_57’. The range  of the scale is arbitrary (e.g. 1-10, 0-99, 1-1000), but should have enough levels to  distinguish between  question difficulties.
The Testing Process
The Adaptive Test activity is configured with a fixed starting level. The test will  begin by presenting the test-taker with a random question from that starting level.  As described in Linacre (2000) , it often makes sense to have the starting level  be in the lower part of the difficulty range so that most test-takers get to answer  at least one of the first few questions correctly, helping their moral.
After the test-taker submits their answer, the system calculates the target question  difficulty it will select next. If the last question was answered correctly, the next  question will be harder; if the last question was answered incorrectly, the next  question will be easier. The system also calculates a measure of the test-taker’s  ability and the standard error for that measure. A next random question at or near  the target difficulty is selected and presented to the user.
This process of alternating harder questions following correct answers and easier  questions following wrong answers continues until one of the stopping conditions is  met. The possible stopping conditions are as follows:
there are no remaining easier questions to ask after a wrong answer
there are no remaining harder questions to ask after a correct answer
the standard error in the measure has become precise enough to stop
the maximum number of questions has been exceeded
Test Parameters and Operation
The primary parameters for tuning the operation of the test are: