Statistics and Public Policy Debate

[Chris Lloyd]

(This post is based on the Belz lecture I gave on October 14th. The slides are HERE. The most important part of this post is the last section. I would be very grateful to have some ideas on the questions I pose there.)

There are plenty of public intellectuals that are prominent in commenting on issues that directly inform public policy. The best known are probably Tim Flannery and Peter Singer. Another is Ross Garnaut who is routinely asked for comment on any issues that relate to climate change and resource rent taxes. Historians Henry Reynolds, Keith Windshuttle and Robert Manne are well-known for their internecine battles that have become milestones in the so-called history wars. Andrew Leigh (now the federal member for Fraser) and Joshue Gans are economist who write opinion pieces or appear on ABC radio on a weekly basis. There is not a single academic statistician or data analysts that contributes regularly to public debate. Why?



First, I wanted to*point out*that not all public policy issues are ones where expert academics will carry the day. There is no way that an expert can convince you logically that gay marriage should be legalized. This is a value judgment. Public issues differ in many ways but one critical dimensions is the extent to which they depend on objective and quantifiable facts rather than subjective values. Peter Singer mainly lives down the value-laden end of the spectrum. Experts like him can still make a big contribution to debate on value dependent issues but their contribution is nor decisive.

At the other extreme are issues which depend critically on objective and quantifiable facts, usually embodied in a set of numbers. They final decision may also depend on values but only after you know the numbers. The empirical evidence may be biased in various ways and perhaps cannot be taken at face value. This is where statisticians (and econometricians who currently inhabit this public space) can be immensely helpful. Once the facts have been objectively assessed, public debate can move on to values and trade offs.

I thought I would list a few examples of where objective, expert assessment could have been very helpful to public discourse.

Did Rudd’s*stimulus work?

The Rudd government spent about b$10.4 in discretionary stimulus spending in 2009. Many other countries also applied stimulus packages of varying size. So the question becomes

Did those countries that applied larger stimuli see a better economic outcome, after adjusting for other relevant variables?

Treasury had a go at this question and used IMF data on stimulus and GDP growth to produce a significant regression for 11 countries including Australia, see their 2010 budget statement papers #2. The selection of these 11 countries was criticised by an RMIT economist and treasury were publically accused of selective use of statistics and data snooping.

Statisticians would have a lot to add to this debate. We could use cluster analysis or other methods to select peer countries for Australia, we could include control variables in the regression and, most importantly, we could try to stand above the adversarial political environment that the issue was discussed in.

How many died in Iraq?

In 2006, a study published in the Lancet estimated that 655,000 people in Iraq were dead who would not have been dead if the war had not been prosecuted. Not surprisingly, the issue was immediately politicised. The gory details of the sampling and methodology used weres never going to get a full airing in the media, so it degenerated into a “who do you believe” issue.

Casualties in war zones are difficult to count because of the chaos on the ground and the lack of centralised government infrastucture. The Lancet study used a sampling method known as multi-stage cluster sampling. They used a log-linear model to estimate the excess number of deaths. If this is not an issue where statistical experts could contribute to public debate, then I don’t know what is.

I had a close look at the Lancet paper myself and find that it is lacking in important, indded to my mind essential,*detail. The exact protocol for selecting the cluster of houses is not given and bearing in mind the potential for contamination and bias of reported deaths this is a huge issue. The details of the statistical model fitted are also lacking, though I am*advised by people in this area that the severe space restrictions imposed in large prestigious journals means that minor details like how you analysed the data(!) are often omitted. On the basis of about a full day of reading and thinking, I came to*the view that the the paper should not have been published without much more detail. For an issue of this import, the Lancet should just devote a few more pages of their journal! That is not to say the 655000 figure was wrong, nor that the research was necessarily unsound. But the published paper was not sufficiently transparent that it could be properly assessed by the peer community (or by me).

The population of Australia.

This is an issue that is of continuing interest. About 10 years ago Peter Costello was telling us we need to have more babies while in the most recent election both parties were saying they wanted a smaller population (though neither had a credible plan to achieve this). The issues with population are (1) total population and its effect on infrastucture and the natural environment, (2) the age distribution and how it affects GDP per head, and (3) cultural diversity.

You would think that demographic projections would be uncontroversial enough that the facts could be on the public table and there would be some consequent consistency on public policy and discourse. For the Belz lecture, I ran a few projections of my own. I am not convinced that statisticians should get into the third issue of cultural diversity. It is too difficult to measure (apart from silly measures such as the mean number of Australian ancestors people have) and whether you consider diversity a good thing, a bad thing, or something that is optimised in the middle, which*is mainly a value judgment.

So I just focused on total population and various age dependency ratios. I measures dependency ratios in terms of an equivalent retirement age (ERA) to maintain ratios at 2010 levels (which I think might be novel?). At the end of the day, I came up with three propositions which I think all demographers would confirm and which have some pretty obvious implications.

First, the population is set to age drastically over the next 30 years. This is a consequence of having fertility rates of 3.5. post war and then changing to fertility rates lower than 2. It is largely inevitable, and we would be better off debating how we are going to cope. Encouraging later retirement and savings rates are pretty obvious starters. Second, to the extent that we can affect the number of workers per elderly person, either higher fertility or migration is equally effective. And we can only reduce the ERA from 73 (which is where it will be in 2050 with zero population growth) to maybe around 71. Thirdly, if one includes the youngest age group of 0-15 as dependents as well as the retired, then the ERA can be somewhat reduced by a combination of lower fertility rates and higher immigration. The reason is that fertility injects babies into the population who have to be fed and educated for 15 years whereas immigrants tend to be overwhelmingly in the more productive age ranges. Low fertility and highish immigration was pretty much where we were 10 years ago before politicians started suggesting that the aging population was a problem that could be fixed.

There were several other public issues that I could have discussed in the talk but didn’t. Did Howard’s gun buy-back have an effect on homicide and suicide rates? How should we measure the performance of schools? Does prison reduce crime? How much fo the falling road toll is due to random breath testing. Should the legal limit be further reduced or have we already hit diminishing returns? And let’s not forget that endangered polar bear in the room climate change.*

How can we be involved?

There is*nothing to stop individuals becoming regular media contributors on statistical issues, in the same way that Andrew Leigh and Joshua Gans do. The problem is that there is no incentive to do so, especially for young academics. Andrew Leigh was able to rise to ANU Professor based on innovative and careful empirical econometrics. I do not believe that statisticians can do the same. There is a cultural bias in the academy against applied as opposed to methodological research.

I wonder then whether we might consider setting up some kind of panel of experts. The advantages of this are (1) The authority of a panel. The public are already familiar with letters signed by panels of experts. The idea is that the political biases in such statements have been largely excised through*the process of getting unanimous panel agreement, (2) Closely related to this is that critical analysis from many heads is likely to lead to a more reliably assessment than that of an individual. Personally, I would be pretty nervous about publically pronouncing on the Iraq body count without the robust input of my peers.*(3) The workload can also be spread across several people. I envisage that perhaps one member of the panel would write the report, and then have it critically examined by the others.

?



Get More from the original blog...