Professional Philosophers’ Susceptibility to Order Effects and Framing Effects in Evaluating Moral Dilemmas
Fiery Cushman and I have a new paper in draft, exploring the question of whether professional philosophers' judgments about moral dilemmas are less influenced than non-philosophers' by factors such as order of presentation and phrasing differences.
We recruited hundreds of academic participants with graduate degrees in philosophy and a comparison group of academics with graduate degrees in other fields. We gave them two "trolley problems" and two "Asian disease"-type framing effect cases.
We presented the trolley problems either with a Switch case first (the protagonist saves five people by Switching a runaway trolley onto a side track where it kills one), followed by a Push or Drop case (saving five by Pushing one person into the trolley's path or by Dropping him into its path); or Push/Drop first, followed by Switch. For each scenario, participants rated the protagonists' choice to kill the one person to save the five others, using a 7-point scale from "extremely morally good" to "extremely morally bad".
Our previous research suggests that non-philosophers are much more likely to judge the Push case and the Switch case equivalently when Push is presented first than when Switch is presented first. On some views of philosophical expertise, philosophers' judgments about these cases should be less dependent on order of presentation than are non-philosophers' judgments. In other words, philosophers, due to their familiarity with scenarios of this type and their expertise in applying moral principles to them, should have more stable opinions, less influenced by order of presentation. We wanted to see if philosophers with prior familiarity with the cases, or self-reported expertise in the area, or self-reported stability of opinion, would show smaller order effects. We also wanted to see if we could reduce order effects by enforcing a delay before responding during which we encouraged participants to reflect carefully on different versions of the scenario and different ways of phrasing the scenarios.
We were unable to find any level of expertise at which the order effects were detectably reduced. Nor did adding a reflection condition appear to reduce the order effects. This figure shows the rates at which Switch was rated as morally equivalent to either Drop or Push on the 7-point scale:
[click to enlarge]
The order effect, as indicated by the differences in height between the black and the gray bars, is basically the same for philosophers and the non-philosophers, even in the "reflection" condition.
This next figure breaks the results down by degree of self-reported expertise among philosopher respondents:
[click to enlarge]
Note that at no level of expertise does the order effect appear to be reduced: not among philosophers reporting being professors with a specialization in ethics, nor among philosophers reporting having a "stable opinion" about trolley problems of this sort. If anything, the trend appears to be toward larger order effects with increasing expertise.
Some of the most famous results in psychology are the Tversky-Kahneman "loss aversion" framing effects. Participants are asked to imagine that an unusual disease will kill 600 people if nothing is done and then given a choice between two programs: On Program A, 200 people will be saved. On Program B, there's a one-third probability that 600 people will be saved and a two-thirds probability that no people will be saved. When the decision is framed this way, in terms of the number "saved", most people favor the non-risky Program A. When what are (purportedly) the exact same options are presented in terms of how many will die (400 will die vs. one-third probability that none will die and two-thirds probability that 600 will die), respondents tend to favor the risky Program B.
The results:
Percentage of philosopher respondents recommending the risky Program B, by framing, level of expertise, and order of presentation:
[click to expand]
As is evident from the figure, our philosopher respondents showed very large framing effects (similar to those of our comparison group and similar to the effect sizes seen in other studies with non-expert populations) -- again up to very high levels of expertise, including self-reported expertise on framing effects and self-reported stability of opinion about framing effects. To see this, look just at the black bars above, ignoring the gray bars.
Philosophers also showed large order effects when they were presented two slightly different framing-effect scenarios, either die-frame followed by save-frame or save-frame followed by die-frame. To see this, compare the adjacent pairs of black and gray bars above.
Full manuscript in draft here. Comments welcome!