Stimulus sampling and other recommendations for assessing domain-general processes of attitude formation through exploration: Reply to Ruitsch, Shook, and Fazio (2020).

In response to Ruitsch et al., 2020, British Journal of Psychology, we propose that the assessment of domain-general ideological differences requires systematic stimulus sampling. We argue that there is currently no evidence that the 'neutral' BeanFest assesses domain-general ideological differences and that Ruitsch et al., 2020, British Journal of Psychology findings do not address the mechanism(s) underlying our findings.

We are grateful to Ruisch et al., 2020;hereafter, RS&F) for their critical comments on our article (Fiagbenu, Proch, & Kessler, 2019), in which we showed that ideological differences in exploratory behaviour and attitude formation are more flexible than suggested by previous research (Shook & Fazio, 2009;hereafter, S&F). RS&F raised two concerns about our study. First, they argue that we have mischaracterized S&F's study. Second, they claim that their findings identify the mechanism underlying the attitude reversal we demonstrated in our study. We respond to these critiques below.
To start with, we commend RS&F for accurately identifying the similarities and differences between S&F's study and ours. We agree with RS&F that the goal of our study was to demonstrate that multiple contextual features of complex stimuli (e.g., beans and stocks) can influence exploratory behaviour and attitude formation processes. We, however, disagree that the BeanFest paradigm is 'neutral'. We wish to draw RS&F's attention to the fact that S&F referred to the graphical stimuli as 'good' and 'bad' beans (S&F; p. 997). Exploration of 'bad beans' can be potentially interpreted as negative and threatening, which may have influenced S&F's findings.
This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited. However, RS&F seem to dismiss the potential impact of explicit references to food in the BeanFest task. They argue on a priori grounds and without systematic evidence that participants in S&F's did not (or even could not have) construed the 'neutral' BeanFest task as a food foraging task. Subtle cues and unconscious processes influence judgement and decision-making (Newell & Shanks, 2014). Also, task-irrelevant features in multiplecue judgement tasks may influence internal judgement processes and may cause participants to simulate experimental tasks differently from the external task environment intended by researchers (Fiedler, 1996;Lagnado, Newell, Kahan, & Shanks, 2006). Our recommendation therefore is that the 'neutral' BeanFest should be compared with other version(s)food and/or non-foodin order to resolve the issue of whether task framing may have influenced S&F's findings.
Regarding their second concern, RS&F suggest that our findings 'simply stem from preexisting differences in attitude towards the target stimuli' (p.4). This explanation presupposes that conservatives (vs. liberals) have more positive attitudes towards stocks and more negative attitudes towards beans. However, their findings show that conservatives (cf. liberals) have more positive attitudes towards both beans (slightly) and stocks. A plausible implication of this finding is that conservatives (cf. liberals) should be more exploratory towards stocks and slightly exploratory towards beans. Our findings, in contrast, showed that conservatives (cf. liberals) are more exploratory towards stocks but less exploratory towards beans. Clearly, RS&F correlational evidence do not sufficiently explain the mechanism(s) behind the attitude reversal in our study.
RS&F, however, interpret their evidence differently. They pre-emptively dismiss the potential implications of their own findings and proceed to declare beans as neutral stimuli. The approach on which this declaration is based is unsatisfactory. In order to assess the neutrality of a stimulus, we suggest that several attributes of the stimulus must be taken into account because stimuli are defined by multiple features (e.g., Fiedler, 1996). No one would seriously contend that positivity/negativity dimension is the only approach to assess attitudes towards complex real-world stimuli like beans. For example, evaluation of several features of beans (e.g., shape, taste, variety) may influence attitudes in different ways.
Furthermore, RS&F argue that strong positive pre-existing attitudes towards stocks overcame domain-general differences in our study, whereas slight positive pre-existing attitudes towards beans did not override domain-general differences in S&F's study. Their post-hoc explanation begs the question: does such a 'general tendency' exist? We argue that such domain-general processes, if they indeed exist, still await empirical demonstration.
Finally, assuming that beans are 'neutral', there is no reason to believe that only beans should be used to assess exploratory behaviour. S&F's study is therefore selective and precludes generalizability of their findings to other 'neutral' stimuli. Our recommendation therefore is that stimulus sampling may resolve the issue of whether stimulus features potentially influence attitude formation processes. For instance, given that RS&F's supplementary findings suggest that potatoes (r = À.09, p = .07), pasta (r = .06, p = .27), and money (r = .03, p = .54) are neutral stimuli, a starting point will be to examine whether S&F's findings emerge across these stimuli (and others). Another possibility is that, in order to retain the 'neutral' nature of the task, references to taskirrelevant cues should be avoided, so that participants would engage with only the abstract graphical objects as the researchers originally intended (but see, Wilson, Ausman, & Mathews, 1973).
To be sure, we do not seek to question the potential merits of research on attitude formation through exploration. Our contention is that stimulus sampling may be a necessary procedure to illuminate the supposed domain-generality of the BeanFest task, in order to address how attitudes are formed through exploration of 'neutral and novel' realworld contexts. Many so-called domain-general processes of popular psychological phenomena have been shown to be malleable following stimulus sampling (Bahn ık & Vranka, 2017;Elad-Strenger, Proch, & Kessler, 2020;Gigerenzer, 1991;Gray & Keeney, 2015;Proch et al., 2018). The BeanFest program of research (Fazio, Pietri, Rocklage, & Shook, 2015) seems to join these studies, providing a cautionary tale for researchers seeking to understand domain-general processes of attitude formation towards novel stimuli tomind the gapand apply stimulus sampling. Without stimulus sampling, ecological validity may not be achieved even with the most sophisticated, well-validated, and replicable paradigms.