Free beta access. 50 free AI tokens each week. Optional top-ups available. See what is included
Opening Word document…

The data they studied isnt conclusive – the research subjects were too broad and they cannot account for social priming of subjects

Nitsch et al 2022 - Comparative Psychology, Institute of Experimental Psychology, Heinrich-Heine-University Düsseldorf, Düsseldorf, Germany

Felix J. Nitsch, Luca M. Lüpken, Nils Lüschow, and Tobias Kalenscher July 26, 2022 “On the reliability of individual economic rationality measurements” Psychological And Cognitive Sciences 119 (31) e2202070119 https://doi.org/10.1073/pnas.2202070119

A more general point we want to raise here is the validity of economic rationality as a psychological construct, which is a prerequisite to valid measurements (39). Strictly speaking, economic rationality describes whether or to what extent a set of choices can be described by a utility function. In the recent literature, however, economic rationality of a finite set of choices of an individual has been compared with decision quality (11, 29, 40), policy responsiveness (41), or variability in the neural computation of value (27) based on face validity and correlational evidence,** all of which are arguably related but not identical constructs. Given this lack of a clear definition of the psychological construct to be measured and lack of evidence of validity for widely used measures, advances in psychological theory and measurement appear necessary. A recent, particularly promising approach for the applications outlined here are generative models,†† which can serve to formalize psychological constructs and increase the reliability of behavioral measurements (43, 44). The theoretical basis to inspire the development of such models could be provided by cognitive science: past studies have shown that cognitive skills (e.g., executive control, working memory, and intelligence) pose a common factor of many aspects of decision making, including choice consistency (45–50). Limitations. A limitation to our assessment might be that the upper limit of effective trials considered for our reliability assessments was 80, which is below that of a few particularly high-powered studies (e.g., [27] used up to 108 trials). Optimistically, it could be possible to improve the reliability of rationality measurements by increasing the number of trials in behavioral experiments to further reduce measurement error. However, a high number of trials (i.e., at least more than 80 trials) comes at the cost of practical feasibility in many studies and the risk of increasing fatigue due to prolonged measurement durations, which in itself might bias the measurement (ranging from relative changes in preference to qualitative changes of decision strategy). To give a benchmark, in study 2, the number of participants indicating that they were “very much fatigued” tripled from after 20 to after 40 trials (after measurement 1: n = 10; after measurement 2: n = 29; SI Appendix, Fig. 2). Another potential limitation is that we could not replicate the finding of Breig and Feldman (31) that allowing participants to revise their choices leads to an increase of revealed preference consistency in the choice set. We acknowledge that, as the authors also demonstrate in their paper, the effectiveness of such an intervention depends on the specific configuration of the choice interface. Hence, due to the ineffectiveness of the intervention, we cannot rule out that a more effective intervention could increase the reliability of rationality measurements. Lastly, we acknowledge that while the qualitative results for each dataset are similar, there is some variability in the quantitative reliability estimates, which could be driven by the heterogeneity of the included datasets.

Sign in to recut, update, or save this card.