Generalizability: Considerations

Back Back to Generalizability

There are three types of generalizability that interact to produce probabilistic models. All of them involve generalizing a treatment or measurement to a population outside of the original study. Researchers who wish to generalize their claims should try to apply all three forms to their research, or the strength of their claims will be weakened (Runkel & McGrath, 1972).

In one type of generalizability, researchers determine whether a specific treatment will produce the same results in different circumstances. To do this, they must decide if an aspect within the original environment, a factor beyond the treatment, generated the particular result. This will establish how flexibly the treatment adapts to new situations. Higher adaptability means that the treatment is generalizable to a greater variety of situations. For example, imagine that a new set of heuristic prewriting questions designed to encourage freshman college students to consider audience more fully works so well that the students write thoroughly developed rhetorical analyses of their target audiences. To responsibly generalize that this heuristic is effective, a researcher would need to test the same prewriting exercise in a variety of educational settings at the college level, using different teachers, students, and environments. If the same positive results are produced, the treatment is generalizable.

A second form of generalizability focuses on measurements rather than treatments. For a result to be considered generalizable outside of the test group, it must produce the same results with different forms of measurement. In terms of the heuristic example above, the findings will be more generalizable if the same results are obtained when assessed "with questions having a slightly different wording, or when we use a six-point scale instead of a nine-point scale" (Runkel & McGrath, 1972, p.46).

A third type of generalizability concerns the subjects of the test situation. Although the results of an experiment may be internally valid, that is, applicable to the group tested, in many situations the results cannot be generalized beyond that particular group. Researchers who hope to generalize their results to a larger population should ensure that their test group is relatively large and randomly chosen. However, researchers should consider the fact that test populations of over 10,000 subjects do not significantly increase generalizability (Firestone,1993).