On 3rd Nov we had a seminar presentation by Johannes Haushofer on a RCT in Kenya. In this experiment, some households in villages is Western Kenya were given unconditional cash transfers of either USD 404 or USD 1525. The researchers measured psychological wellbeing and found, unsurprisingly, that the lucky ones were happier and that their unlucky neighbours were unhappy. The paper is aptly titled “Your Gain is my Pain”. During the seminar everybody appeared to be trying to understand the identification strategy and thinking of ways to criticise it.
I was just overwhelmed by a sense that this type of research should not be done at all. No matter how clever the identification strategy is. Am I the only one to think that is not ethical dishing out large sums of money in small communities and observing how jealous and unhappy this makes the unlucky members of these tight knit communities? In any case, what do we learn from a study like this? The results indicate that we should not single out households and give them large sums of money. So I hope this puts an end to unethical RCTs like this one.
The picture is taken from http://www.wspus.org/2015/07/how-progressive-ideas-protect-inequality/
This short reaction to what is just one example of the many RCTs conducted in Development economics, points out to the need for a reflection on the ethical specificities of RCTs in dvp economics. I have started the discussion in my my paper “The ethics of New Development Economics: is the Experimental Approach to Development Economics morally wrong?”, http://jpe.ro/pdf.php?id=4998), where I identified 6 major problems potentially raised by ethically ill-designed dvp economics RCTs:
1. The ‘hazardous calculus problem’, or the problem of problematic unintended (or even worse in this case, intended) consequences.
2. The ‘randomization problem’, which brings the issue of treating unequally
equal people
3. The ‘consent problem’, which relates to the obligation of respecting individual
autonomy
4. The corollary ‘instrumentalization problem’, which is the actualisation of
Kant’s deontological interdiction to treat persons merely as means.
5. Raised by the first 4 problems, the ‘accountability problem’ raises the question of the responsibility of researchers in case their experiments have damaging consequences.
6. Finally, the ‘foreign intervention problem’ is that of foreign intervention in the
domestic affairs of an underdeveloped (and therefore weak) country.
The paper discussed here, which I don’t know, seems to hit hard the first two problems at least.
The discussion has now been picked up and continued by several authors including Macartan Humphreys (“Reflections on the Ethics of Social Experimentation”), but remains marginal. This is unfortunate, as I believe the stakes are high: not only ethical standards are important in themselves, but if well-thought they drive the scientificity of research higher and in this sense prevent unnecessary reasearch to take place – unnecessary in both the ethical and scientific senses.
Dr. Stephane Baele, Exeter.
Prejudice masquerading as science. If this is not prejudice but curiosity, I would like to assure this experimenter that at the molecular level, there is no fundamental difference. We all bleed the same. “Your Pain is my Gain” is a well known phenomenon, both in the Western World and other places. In fact, during the discourse in 2002 following the corporate governance scandals and excessive executive compensation typified by the Enron scandal, Arthur Anderson and Global Crossings, a commentator noted that most executives thought they were well paid until they visit the club house and learn of what others are earning. Then they feel cheated and underpaid. Interestingly, that commentator labelled the feeling of relative deprivation by the executives a “phallic thing.” Robert Frank raises a similar issue in his contribution on Economics, Values and Organization. His piece is titled “Social norms as positional arms control agreements.”
My point is that the purpose of the Kenya experiment is to make a caricature of folks at home. That is so reminiscent of the starving kid, looking dirty and being attacked by flies picture of aid campaign for Africa in the 90s. The more gruesome, the higher the likelihood of attracting donors. It is the well-known Samaritan’s Dilemma in the aid literature. It was only a matter of time for it to creep in, masquerading as scientific research, into development economics (reloaded).
So, my final advice to this RCT scientist is watch the movie, “Trading Places” starring Dan Akroyd and Eddy Murphy if you are still unconvinced about universality of basic human behavioral attributes. I work in the Middle East. I know for certain that many students are less distressed with a C if the competitor scored a D but are unhappy with an A- if the other gal scored an A. We are all the same, capable of the same basic instincts, products of serendipity and other circumstances.
Great blog and great comments from Stephanie and Melvin.
It seems that some RCTs either choose to wilfully ignore human behaviour or have been unable to incorporate learnings and practice from the fields of psychology, sociology, behaviourial economics, etc. that give us profound insight into how people behave under specific conditions. I say this because this RCT seems to have been based on an assumption that poor people in Kenya might not exhibit the same behaviours as anyone else. This to me is strange – the investigators are almost trying to ‘test’ and ‘prove’ the reaction. I wonder how the findings would actually be useful to policy and projects. I would also wonder what long-lasting impact this type of research would have on social relationships in the communities in the future.
There is more debate of this RCT on the World Bank website by Berk Ozler
http://blogs.worldbank.org/impactevaluations/unbearable-subjectiveness-wellbeing
and David McKenzie (Lead Economist, Development Research Group) who thinks that “this is important research, although an alternative take is offered at the CSAE blog who argues that it is unethical (I link to this not because I agree, but because I often hear vague complaints about RCTs being unethical, but it is hard to find concrete examples of specific studies for which this concern is raised).”
http://blogs.worldbank.org/impactevaluations/weekly-links-november-6-5-years-nudging-peer-effects-not-enough-news-new-zealand-again-and-more
I agree with Stephane’s list, but perhaps the simple ethical rule should be to ‘do no harm’. To be ethically justifiable an RCT should be designed so that nobody is made worse off by the treatment; one is testing if the treatment is Pareto improving. The RCT mention clearly fails, and in addition reports an effect that everybody (else) knows, as Milton observes – not only unethical but also unnecessary. Most RCTs have an inherent tendency to fail this rule: if the treatment makes the treated better off, the untreated will (if aware of this) feel worse off. Ironically, unsuccessful RCTs, where the treatment has no significant effect, may pass the rule. One way to comply with the ethical rule is to ensure that the untreated are kept unaware of what happens the treated (which implies types of RCTs that meet ethical standards). An alternative is that if the treatment is inherently beneficial, an ethical design would allow the untreated to be treated at a later stage.
I agree with Stephane’s list, but perhaps the simple ethical rule should be to ‘do no harm’. To be ethically justifiable an RCT should be designed so that nobody is made worse off by the treatment; one is testing if the treatment is Pareto improving. The RCT mentioned clearly fails, and in addition reports an effect that everybody (else) knows, as Milton observes – not only unethical but also unnecessary. Most RCTs have an inherent tendency to fail this rule: if the treatment makes the treated better off, the untreated will (if aware of this) feel worse off. Ironically, unsuccessful RCTs, where the treatment has no significant effect, may pass the rule. One way to comply with the ethical rule is to ensure that the untreated are kept unaware of what happens the treated (which implies types of RCTs that meet ethical standards). An alternative is that if the treatment is inherently beneficial, an ethical design would allow the untreated to be treated at a later stage.
One of my students saw this interesting thread and sent it my way – I hope you don’t mind me chiming in.
This design does capture many potentially problematic features of new experimentalism. Here are a few things that struck me:
The experiment seems to use a third-party to avoid any human subjects rules, especially informed consent. This is entirely legal. Yet is is not clear how the randomization servers more than the researchers’ agenda. Is this just a case of hiding behind a third party to skip ethics rules?
The amounts seem large, and very consequential for subjects’ well-being.
There is no discussion of ethics in the paper.
We had a conference at UCSD on these topic about 2 years ago, with proceedings now forthcoming. Some of the main conclusions:
“Do no harm”, as per another poster, is a great starting point.
Informed consent solves many problems. Why not invite people to be in a study, rather than just manipulate them to advance your research agenda?
Combining these first two points, why not debrief subjects post-study and ask them how they feel about being subjects in the experiment?
Discussions of ethics should be part of the research enterprise. Write-ups should provide details on any potential issues, how they were mitigated, and the nature of the relationship with the third party. And peers should ask questions about ethics – just like you are doing here.
While the objective of research is to create knowledge generally, I believe that research in development economics should create knowledge that can inform public policy and enhance human development. The Kenya research, in my opinion, does not provide “knowledge” that can improve the human condition. In addition to the fact that the “research,” as designed and carried out in this Kenyan village, raises a lot of questions regarding the way researchers treat vulnerable groups and communities, it also undermines the work that many Western-based researchers and institutions are doing to improve public policy in developing countries, including Africa. Perhaps, more important is the fact that the design of these so-callled “experiments” has the potential to destabilize this rural village and create problems that can endanger village governance and impede human development.
Does such research advance efforts to improve living conditions for these research subjects or others similarly situated? If no, then, what is the objective of engaging in such exercises?
Such a experiment is bound to create problems but what about the real thing not experiments. Some NGO UCT’s have an evaluation process on who will receive the UCT and who will not. The evaluators go round the village doing their things. Then one day the phones start bleeping for those who have received the UCT.
It quickly gets round who has got the UCT and who has not and then the questions start, e.g. how can someone with a job get a UCT and someone without a job not get a UCT. It does start the jealousy factor.