OMNILIBRIUM
  Rational Discussion of Controversial Topics
Economics Education Ethics Foreign Policy Government History Politics Religion Science


Problems with objective evaluation of government policies. Part I. The unemployment data paradox

melian          2 April 2016 08:10 AM


A scientific revolution which completely transformed our civilization was fundamentally based on a very simple idea. Any theory, no matter how elegant or attractive, should be discarded if it fails experimental testing.

It has been often suggested that the same idea should be applied to politics. Any ideological principle, no matter how elegant or attractive, should be discarded if its implementation consistently fails to produce the desired results. Likewise, all politicians who perform poorly in the office should be replaced regardless of how noble are their goals.

Unfortunately, applying this idea to politics encounters two major obstacles. First, being a much more emotional field than any of the hard sciences, politics makes rational decision making process extremely challenging even for highly intelligent people. Second, even in absence of strong emotional biases objective evaluation of government performance is a non-trivial task.

As an example, consider the problem with the US unemployment statistics. According to the current president supporters, one of his biggest achievements was to restore the unemployment rate to its pre-recession levels.







According to his opponents, who look at the employment rates among the entire working age population (see the graph below) there has been almost no recovery.







The explanations for the discrepancy between the two graphs fall in two distinctly partisan groups. For instance, it is possible that with the stock market recovery more older people can afford early retirement. With growing government education subsidies more young people can extend their education. With rising salaries of their spouses more women can afford to stay home to raise their children. Alternatively, it is possible that most people who dropped out of labor force would have preferred a well-paying job but with no good employment opportunities were forced to take the alternatives (going into an early retirement, getting another academic degree etc.).

Do any of the readers know how to decide which explanation is correct?




Would you like to read similar articles in the future?
      
-2 -1 0 +1 +2
 

                  Post Comment                   




Recommended for You Optimates Populares Centrists

Show comments            Sort by        



aliad 7 April 2016 12:08 AM
72%

The correct scientific way to determine the affect of the policy should be to reset time to 2006 and then run forward with a different policy to see if we get different results. And then do that a dozen more times for each policy to see if the experimental results are repeatable. Then do the how thing a score more times checking the affects of other variables to see how sturdy our results were. Of course we can't re-set time or get data from other many-worlds, so seeing the effects of policy clearly is going to be very hard.
Another issue is that the explanations offered for the drop in labor market participation aren't necessarily mutually exclusive. Lots of people would love to make tons of money doing their dream job. ( example I'd love to make 80k a year test playing video games.) But it's not automatically a failure of the economy that it does not always provide that. Likewise I can see going back to school for an advanced degree might be enticing if I got a full ride including care for my dependents. Most people don't get either perfect option. That get a range of imperfect possibilities and choose which to pursue based on multiple factors. Finally both these graphs are of percentages but nether is shown from 0 to 100. We need to keep in mind that these are fairly subtle effects given how much noise there is on the causation side.

There's only one thing I can think of that might help solve this.
An X-prize for predictive economic modeling.
Contestants would have to submit an computer model that was designed to tell you things like the labor partition rate or the unemployment rate a year or two in advance. They can ask the administrators of the prize to feed the model certain limited policy data, like what the Fed target rate is or what the dollar value of the stimulus is, but other than that they can't tweak the model once the submission deadline is passed. A model that can regularly give correct predictions will win the prize.
I'd give it only a 40 percent probability that such a contest find a winner in its first 15 years. Even then the model would soon be destroyed by the observer effect.


stars0
Reply



ChristianKl 4 April 2016 03:48 PM
65%

You can seldom do objective evalaution of government policies based on n=1. That's just not how drawing correct inferences about causality in complex systems works.


stars0
Reply


melian 5 April 2016 04:24 AM
73%

What does n in your comment stand for?

stars0
Reply


ChristianKl 11 April 2016 05:27 AM
65%

If you have an adoption of a policy in one country than you have n=1. If randomly half of the states adopt the policy and half don't you have n={number of states}.

stars0
View Replies (2)
Reply


ChristianKl 11 April 2016 05:39 AM
61%

To add to my last comment, this isn't rocket science. It's causal interference 101. If you get this wrong it's seems like strong evidence of mindkilling effects preventing you from reasoning about his domain.

stars0
Reply


VoiceOfRa 4 April 2016 09:54 PM
65%

So, how is one supposed to evaluate a politician's record?


stars0
Reply


ChristianKl 5 April 2016 04:16 AM
65%

That's an interesting question. On what basis do you think you are supposed to evaluate a politician's record?

Don't mix deontological issues like "what I'm supposed to do" with epistological one's like "how do I know what's true". If you don't mentally distinguish the two, you are going to do a lot of reasoning errors.

stars0
Reply