Difficulties also remain for the most well-standardized instruments. While the CARS has been repeatedly shown to produce autism categorizations much like diagnoses, the items on the scale no longer reflect current diagnostic criteria. The ADI-R and the ADOS produce operational categories that fit with current conceptualizations of autism, but they require training and are intended to be used by experienced clinicians. The ADI-R is also quite lengthy, taking about 2 hours to administer. Standardization samples for both instruments are small, though replications of their diagnostic categorizations have been good (Yirmiya et al., 1994; Tanguay, 1998). Neither provides adequate discrimination between autism and other autistic spectrum disorders, though the ADOS makes a first attempt to do so. Thus, these instruments are important in providing standards for research, but their contributions to educational practice will require training of specialists (both in and outside educational systems) and perhaps modification of the instruments.
To interpret the results of early intervention research and to conduct some of the sophisticated analyses described below, it is important to understand the characteristics of the participants in the studies. As mentioned above, heterogeneity in child characteristics is nearly as much a defining feature of autistic spectrum disorders as are the DSM-IV criteria. Children with the same diagnosis of autistic spectrum disorders, gender, chronological age, and IQ score may well have a range of other different characteristics (e.g., problem behaviors, communication skills, play skills) and may respond differently to intervention treatments. In most research on comprehensive intervention programs using group designs, a limited amount of information is provided about the children participating in the study. Individual intervention practices research often uses a single-subject design; anecdotal descriptions of participants’ behaviors are sometimes provided in addition to demographic information, but such descriptions do not follow a standard format. These limitations are reflected in the small proportion of studies that meet the highest standards for research in internal or external validity, as shown in Figures 1–1 and 1–2 (in Chapter 1), and the greater but still variable proportion that meet the second level of criteria in these areas.
Vaguely described samples pose a problem for both group and single-subject designs. One problem is related to internal validity of the study (i.e., the degree to which a researcher can rule out alternative hypotheses that account for treatment outcomes [Campbell and Stanley, 1963]). Unless specific information about participants is provided, it is impossible to know to whom the results of the study apply. For group design research, there are additional problems. When random assignment to treatment