Making Informed Choices in the Complex World

The human nature is complex. The world has billions of interconnections. But it’s surprisingly simple to understand them.

Here’s a practical question. How can the government improve education? Google returns 600 million answers to this question. Unfortunately, most suggestions can’t help. But we still can find out which of them would help. The MIT Poverty Action Lab did a series of evaluations:


This long page says that half of the programs developed by top experts had no impact on test scores (horizontal lines touch the vertical zero on the left-hand side plot) . Though these programs can be useful for something else, they are money wasted as far as learning itself is concerned. The plot on the right is scarier: it’s cost effectiveness of the programs on a log scale. You can see a 100-fold difference in cost effectiveness of scholarships and information provision—in respect to their impact on test scores. It’s like having two shops in one street: one sells 1 apple for $100, another does 100 apples for the same $100.

Take Google or Microsoft, which question the impact of their actions too. Instead of education, they care about profits. They did similar evaluations to find out that 80–90% of their ideas don’t work.

The world is complex and punishes for unjustified self-confidence. Health care, finance, government, nonprofits employ policies that are supposed to work, but they don’t when tested. And these policies are still in force because, well, someone is already paid for being very confident in them. Besides, recognition of the opposite needs courage and doubts, both of which look harmful to career. Costs and complexity aren’t the problem; evaluations are simple and often very cheap. The aforementioned studies separated out the impact in randomized evaluations, but the choices are many:


And more on them later.

3 thoughts on “Making Informed Choices in the Complex World

  1. […] Actually, anecdotes are popular in economics. However, economics differentiates between case-based proof (which is almost never a proof) and case-based example (which demonstrates what you’ve proved with statistics). Second, yes, identification is a great thing—it shows what strings to pull to achieve socially beneficial results. The paper should have mentioned how many people suffer due to policies invented with anecdote-based reasoning that lacks causal connections. […]


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s