1. B is between A and C

2. C is between A and B

3. A is between B and C

4. A and B are after C

5. B and C are before A

The question is designed to test the ability to reason logically, and most people would choose option number 1. People less able to reason logically might pick any one of the options. Thus, the question appears to select against the less able.

    However, we are accustomed to letters of the alphabet being arranged in linear series, as in the text you are now reading. It is likely that both the person who set the question, and most people who read it, would assume this, probably quite unconsciously. However, if the letters were arranged in a circle, then all five of the options would be correct.

    A person with a more subtle mind would see that the arrangement of A, B and C is not specified. Instead of choosing the first option and quickly moving on to the next question, he/she would have to exclude the circle arrangement (because only one answer is correct) and perhaps other arrangements. His/her progress through the text paper would be slowed. Thus the question penalizes both those less able, and those more able, to reason logically.

    This is not just hypothetical. A study by the author of these web-pages found that, indeed, multiple-choice questions select against the more able students (Forsdyke 1978. A comparison of short and multiple choice questions in the evaluation of students of biochemistry. Medical Education 12, 351-356).

     Most individuals in systems where research peer-review operates have already passed many selective gates (e.g. admission to university, graduate and doctoral degrees, appointment to a university or research institute). Thus, the less able are already likely to have been filtered out. All that is left for the peer-review gate to achieve is discrimination between those of average ability and the more able. This selection is against the more able (i.e. those who can think subtly about difficult problems). If the selection were merely by tossing a coin, the more able would at least have the same chance as being selected as those of average ability.

So, if coin-tossing will not give us at least a 50:50 chance of spotting researchers of high ability, what will? Certainly, it is nothing to do with their appearance. Imagine that you suddenly found yourself by chance on a plane with your country's Olympic team. You would have little trouble picking out the small petite teenager as a likely gymnast, the tall, gangly, male as a likely high-jumper, and the heavy, thick-set male as a likely shot-putter or weight-lifter. 

    If you were similarly on a plane with your country's leading scientific researchers there would be no similar yard-stick. Researchers of talent come in all shapes, sizes, colours and sexes. The one common thread is a deep personal integrity that makes it impossible to engage in the marketing practices that the peer review system, as it currently operates, requires. Biologist Leigh Van Valen said it quite succinctly in 1976:

      The Olympians who share your flight did not get selected to represent their country on the basis of what they proposed to do in the future. They were selected because they had demonstrated that they were the agilest, could jump the highest, and could lift the mostest! Similarly, if you were on a plane with top musicians you would know that each musician was there because he/she had repeatedly and consistently demonstrated his/her musical excellence. 

    Yet, of our researchers, while giving lip-service to track-record, we demand that they market a project that will be assessed by "peers." This requires that they discard projects that are difficult to market even though they believe they are the best projects, and propose projects that are easy to understand and are likely to receive matching funds from the pharmaceutical industry. Scientists of deep integrity can no more do this than the elfin gymnast could have lifted weights, or the weight lifter could have high-jumped! 

    Fortunately, there are some fine medical researchers of high integrity who somehow make it in our strange peer review funding system. But if many medical researchers are selected because of their skills in marketing, rather than in research, should we be surprised that, as pointed out by bioethicist, Jackie Smith (2003; Toronto Globe & Mail. 14th Jan):

And when these same people are called upon as the acknowledged "experts" to advise on matters that require high expertise, do we get the best advice?