Economists don’t do experiments. Or – at least – they don’t do experiments the way that other disciplines do.
In most natural sciences, experiments are carried out under painstakingly controlled conditions, most frequently in a laboratory setting. Whether you are a physicist or a chemist or an electrical engineer, laboratories provide the ideal setting in which to tinker with one variable while controlling for everything else that might potentially bias any observed differences in the outcomes between two groups: an “experimental” group and a “control” group.
For example, if an agronomist wanted to assess whether a particular variety of fertilizer is superior to another in the germination of radish seeds, the agronomist would plant two identical sets of those radish seeds in identical soils in the same greenhouse and then grow them in the same sunlight and water them on identical schedules.
With one difference, and only one difference: One set of seeds – the experimental group – would receive the fertilizer, while the other – the control – would not. If the eventual differences between the outcomes for the experimental group are so distinctly different from the outcomes for the control that they cannot be ascribed to mere chance, then we conclude that the “treatment” of the fertilizer has a statistically significant effect on the germination of those seeds. And researchers grow in their confidence about the result the more distinct the difference, the greater the number of seeds in each group, and the number of times the same result is reproduced. Such replicability of experimental results is frequently referred to as “robustness.”
Economists are quite divided on this year’s prize
Some social sciences perform experiments of their own, most notably psychology. But even the psychologists craft and hone their experiments in much the same way as the natural scientists do: They randomly assign individual human experimental subjects into “treatment” and “control” groups, and then – under otherwise identical conditions – expose subjects in the treatment group to an influence that has been withheld from the control group. Of course, nobody tells the experimental subjects which group they are in. If the treatment groups respond significantly differently than the control group, we conclude that the difference between the groups cannot be ascribed to chance alone: It must be the treatment!
Economists are relative latecomers to the experimental game, but the work of experimental economists has revolutionized the field. As evidence one need look no further than the 2002 Nobel prize in economics, shared by Daniel Kahneman and Vernon Smith, for their pioneering work in experimental economics. Here the economists do their best to match the psychologists in their own laboratory-like experimental design and methodology. Often conducted on college campuses with university students as subjects, these experiments randomly assign the students into control and experimental groups, examining the differences in the outcomes.
But let’s be honest: All authentic economic decision-making takes place in the real world – which is no carefully controlled lab. Life is messy, and every other letter in the alphabet is changing while economists are attempting to deduce whether A causes B – which is why economists are always on the lookout for “natural experiments” in the real world.
What’s a natural experiment? It might be when the legislature in one of two otherwise similar U.S. states – say, Oregon and Washington – legalizes recreational marijuana and the other doesn’t. Voila: a natural experiment. If economists want to know whether such a change affects college graduation rates or birth rates or just about anything else, those economists can look at the differences between the states after the change. Now the numbers will be different, of course, but if they are sufficiently different we can conclude that the policy change had a significant effect.
Which brings us to the recipients of this year’s Nobel prize in economics.
Abhijit Banerjee (MIT), Esther Duflo (MIT), and Michael Kremer (Harvard) shared the award for their revolutionary work in which they conducted their own field experiments on poor people to find out which policies work as expected and which don’t.
Rather than wait around for natural experiments, the laureates identified otherwise similar villages or otherwise similar schools and then treated one as the experimental group and the other as the control. They gave one school more textbooks than the other. They fed meals to the children in one school and didn’t in the other. Or they charged families in one village a higher price for a vaccine than in another.
And then they assessed the results.
While more books and more food didn’t improve learning, more individualized instruction did – especially with the students who were furthest behind
As it turns out, having more textbooks actually doesn’t improve learning outcomes. Being fed a meal at school makes no difference, either. And teachers perform better when they face an annual renewal decision for their contracts rather than when next year’s employment is more certain.
As an economist I can tell you that we adore unexpected, counterintuitive results like the ones for textbooks and meals. And researchers like Banerjee, Duflo, and Kremer identify the source of such outcomes: We make the mistake of treating all the students the same, and we expect the average student to respond positively to treatments like more books and meals.
But students are individuals.
And while more books and more food didn’t improve learning, more individualized instruction did – especially with the students who were furthest behind, whereas more books seemed to matter most to the students who were already doing well.
But I can also tell you that economists are quite divided on this year’s prize.
The more free-market-minded ones are frustrated that economists are again celebrating the economist-as-technocrat. And the others, the technocrats, think this work represents one more step toward “solving poverty.” But it seems we are always about to solve poverty.
I think the truth lies somewhere in between. To the extent that flourishing economies burst forth from fertile soil, it’s worth asking whether economic experiments like these can aid us in understanding how soil that is currently infertile might be enriched. But we should also ask whether some of our current efforts are poisonous rather than enriching.
Textbooks and school lunches don’t matter. What does? And when does our helping hurt? These are worthy questions, indeed.
