The federal No Child Left Behind act mandates that programs paid for by the federal government be supported by scientific research demonstrating their effectiveness in student learning. So far, this mandate has generally been interpreted as requiring the use of comparison groups that have been randomly selected.

It is easy to understand why Congress demanded such evidence. Too often, curricula have been endorsed--and millions of dollars in federal, state, and local money spent--based much more on a program's conformance to then-popular ideological principles than reliable evidence of student learning. It is also easy to understand much of the opposition to the mandate: if done well, it threatens the continued success of many of the most popular educational programs.

It is also easy to understand the preference for randomized research. If properly implemented in the right circumstances, it allows the isolation of the effect of a factor being studied from the many other factors that can affect student learning. For this reason, it is often called the "gold standard" of research.

Currently the major federal effort to evaluate research against the randomized research standards is the What Works Clearinghouse. Currently its web site shows reports on one topic--middle school mathematics--and promises reports for eight others. (Reports on a second topic were posted at the same time as the mathematics reports but have since been removed, apparently due to criticism). A superintendent, school board member or other decision maker hoping to make a better choice of middle-school mathematics program based on these reports is likely to come away frustrated. Only a few studies satisfy the criteria used. Even fewer still showed a statistically significant difference between programs. At best, that decision maker might come away with a healthy skepticism about claims by a publisher that a particular program is supported by research.

We invite papers examining the strengths and limitations of randomized research and of alternative ways of evaluating the effectiveness of educational programs. These papers could take a variety of forms. I suggest the following approaches, but these hardly exhaust the list:

• Surveys of the literature on randomized research and/or alternative approaches in education.
• Examinations or the use of randomized research in other fields, especially medicine which is often put forward as the model for educational research.
• Examination of the effect of teacher, student, or parent attitudes towards alternative programs, especially since the ideal of a double blind test is seldom feasible with educational interventions.
• Critiques of the What Works Clearinghouse approach. (We would particularly welcome papers from those involved in the WWC effort)
• The effect of conflict of interest, since many of the studies have either been sponsored by publishers or the researchers are clear advocates of a particular program.

This is an open-ended call for papers, without a deadline. We hope it will be the start of an ongoing discussion, that will build over time.

All papers should be submitted here. They will be reviewed. As always, reviewers will be asked not whether they agree with the conclusions in the papers, but whether the conclusions are well-supported.

Bruce R. Thompson