The Generalizability of Findings from Laboratory Studies At the heart of this discussion is the distinction between experiments conducted under controlled conditions in artificial laboratory settings and analysis of data obtained from naturalistic environments, such as casinos, lotteries, and markets for betting on horse races or other sports.
While experiments can be carried out under controlled conditions in artificial “realworld” environments, we define a naturalistic environment to be one that “has not been artificially manipulated (i.e., a nonexperimental setting)” (Johnson and Bruce 2001, 266). This distinction is crucial, and there is a long-running debate concerning the relative merits of the two alternative methodologies when employed in experimental psychology (e.g., Ebbesen and Koneˇcni 1980; Hogarth 1981;
Funder 1987, Bruce and Johnson 2003) or economics (Harrison and List 2004; Levitt and List 2007). Levitt and List (2007) pointed out that a critical assumption in experimentation is that results generalize to the broader population. This generalizability or “external validity” has been seriously questioned because of significant variations in observed behavior between laboratory and naturalistic environments (e.g., Ebbesen and Koneˇcni 1980; Koehler 1996).
The factors that have been identified as limiting evidence of biased decision-making in betting markets 489 the generalizability of laboratory experiments (cf. naturalistic studies) include the following: 1. Context: The context in which decisions are evaluated is of central importance. Laboratory environments often present simplified versions of tasks that may be more complex in real-world environments.
Lastly
As a result laboratory experiments may unintentionally omit variables that are influential in the natural setting. It has been reported that significant differences in behavior may depend only on small changes to the experimental conditions (Ayton and Wright 1994), and Glenn Harrison and John List (2004, 1010) noted that although it is tempting to view field experiments as simply less controlled variants of laboratory experiments, we argue that to do so would be to seriously mischaracterize them. What passes for “control” in laboratory experiments might in fact be precisely the opposite if it is artificial to the subject or context of the task.
