Posted by Jarsto in Politics
September 29th, 2010 | 4 Comments »

There are three kinds of lies: lies, damned lies, and statistics.

This saying is commonly attributed to either Mark Twain, or Benjamin Disraeli. I was tempted, given the subject of this post, to look for some statistics on which is the more frequent attribution, but finally decided not to bother. I’ll be introducing one set of statistics shortly, and one is really enough. Judging by the length of this post, more than enough.

I first had the idea for this post a few weeks ago in the run-up to the Pope’s visit to Britain. Although I’m in the Netherlands myself I take in a good deal of British news, and one of the things that struck me was the battle of interpretations over this particular set of statistics.

The poll was executed by ComRes and commissioned by Theos (a pro-religion/pro-Catholic thinktank). Full results are available here. The point I’m going to be examining is Theos’s claim that “whatever the public thinks about the Pope’s visit, they agree with his social teaching”.

The Statistics

The part poll (at least the part of it we’re looking at here) is a series of 12 statements, taking from a papal encyclical letter (Caritas in veritate), with the participant asked to choose either agree, disagree, or don’t know. I’ll start with a quick overview of questions and percentages (percentages don’t always come to 100% due to rounding).

Moral eveluation and scientific research must go hand in hand
Agree: 74%
Disagree: 9%
Don’t know: 18%

An overemphasis on rights leads to a disregard of duties
Agree: 59%
Disagree: 15%
Don’t know: 26%

It is irresponsible to view sexuality merely as a source of pleasure
Agree: 63%
Disagree: 20%
Don’t know: 18%

The natural environment is more than raw material to be manipulated at our pleasure
Agree: 79%
Disagree: 9%
Don’t know: 12%

Investment always has moral, as well as economic significance
Agree: 63%
Disagree: 18%
Don’t know: 19%

The economy needs ethics in order to function correctly
Agree: 79%
Disagree: 7%
Don’t know: 14%

Technologically advanced societies must lower their domestic energy consumption
Agree: 82%
Disagree: 8%
Don’t know: 10%

We must prioritise the goal of access to steady employment for everyone
Agree: 80%
Disagree: 6%
Don’t know: 14%

Poverty is often produced by a rejection of God’s love
Agree: 6%
Disagree: 81%
Don’t know: 13%

The consumer has a specific social responsibility
Agree: 69%
Disagree: 15%
Don’t know: 16%

One of the deepest forms of poverty a person can experience is isolation
Agree: 58%
Disagree: 24%
Don’t know: 18%

Food and access to water are universal rights of all human beings
Agree: 90%
Disagree: 5%
Don’t know: 4%

Interpretation

So those are the statistics, and they themselves I feel quite sure are not lies. While I don’t have the budget (or the inclination for that matter) to travel to Britain and redo the poll myself I would expect anyone doing so to come up with broadly similar figures.

So where are the lies? Well that’s all, quite literally, a matter of interpretation. Based on these figures we can say that the people of Britain broadly agree with the pope on 11 of the statements presented, and strongly disagree with him on 1 of them.

Selective Questioning

But for Theos to be right we have to get from these 11 statements to “the Pope’s social teaching”. And that’s where we run into trouble. Because although these statements are all part of the “the Pope’s social teaching” they are not the whole of the “the Pope’s social teaching”.

This is partly a simple matter of practicality. Having people fill in an “agree, disagree, don’t know” list for Caritas in veritate as a whole (it’s over 30,000 words) would make the results vague at best. And doing it sentence by sentence for the whole things is simply to time-consuming to find a substantial number of volunteers for. And that isn’t even touching on any of the Pope’s teachings not contained in Caritas in veritate.

Let us look for example at “It is irresponsible to view sexuality merely as a source of pleasure”. Now the poll needs pithy statements, but one can wonder what the result would have been had the second half of the original sentence “and likewise to regulate it through strategies of mandatory birth control.” been included. Given the stress on mandatory it might not have mattered a lot, then again it might have.

And if we move on from this sentence and read the whole paragraph it was taken from it’s clear that the Pope doesn’t only disapprove of birth control when it is mandatory. Indeed it is a well known fact that the Pope (and the Catholic Church) don’t approve of voluntary birth control either. Indeed another paragraph speaks of “the procreative meaning of sexuality”.

One might also wonder if the statement (still from the same encyclical) that “… States are called to enact policies promoting the centrality and the integrity of the family founded on marriage between a man and a woman.” would have gotten a very positive response in Britain. Especially if it had been shortened to its core meaning “States are called on not to legally recognise homosexual relationships”.

Straying from the encyclical one could doubt that many British Catholics (never mind people in general) would agree with a statement by the Pope in March of 2009 that “… distributing condoms…. worsens the problem [of HIV/AIDS]“.

This is just a small selection on one topic. But I think it goes to show that claiming (or implying) that people agree with “the Pope’s social teaching” is a whole, is rather misleading based on just these questions.

Uniquely Papal?

Another potential problem with claiming that people agree with “the Pope’s social teaching” is the matter of their reasons to agree with a statement. This one is a bit tricky, so I’ll try to explain. When I say I agree that “It is irresponsible to view sexuality merely as a source of pleasure”. I say so at least partly on a utilitarian basis. Given the risks posed by STDs sexuality is not merely a source of pleasure, but also a source of responsibility: it must be done safely.

As I’ve demonstrated above, this is not what the Pope means by this at all, and certainly not the reasoning he would give for the phrase. His reasons (one would guess) are to be found in the so called “Natural Law theory” often touted by Catholic moral philosophers.

But I have no trouble making this statement as an atheist with a tendency to think in terms of utilitarianism and moral rights theory. In other words, the statement I’m agreeing with is not uniquely papal (indeed far from it). Yet the interpretation (dare I call it spin) that this means I agree with the Pope seems to suggest the statement couldn’t have been arrived at by anyone else. Even though any secular humanist with a liking for utilitarianism could have made the same statement. (Admittedly, human individuality means we would all likely phrase it slightly differently, but the meaning of the statement wouldn’t have to be altered.)

In fact the statement in the poll that comes closest to being uniquely papal (it’s certainly uniquely religious) is “Poverty is often produced by a rejection of God’s love”. This is the one statement that only got 6% of people to agree, and got 81% to disagree.

When the poll first came out I worked out a list, just to amuse myself, of utilitarian ways to arrive at the 12 statements given (I may end up posting this in future if I feel like tidying it up a bit). The rejected statement is the only one I failed on.

It would be easy enough to substitute another statement derived through utilitarianism here, or even the core doctrine that “We should strive to maximise the amount of happiness in the world” and come back with a poll that is at least as strong as evidence that “most British people favour utilitarianism”. Indeed a bit of utilitarian rephrasing on some of the existing statements might well improve the scores further.

Vagueness

The reason we could submit this list (with one substitution) as a utilitarian list is simple. The statements were selected (I suspect deliberately) to be quite vague. They were indeed selected not to be uniquely papal at all.

Because of this the result is vague. Another suggested summary might be “People agree with vague outlines of morality as sketched by pope”. That would still be something of a spin, but closer to the truth. The spin is of course the mention of the pope, because there’s an underlying secret here.

Vague statements of morality tend to work regardless of who uttered them. Because humans are (broadly) similar. We could never have made it this far if most people hadn’t though murder was a bad idea for example. This is why ethical discussions often turn into debates on definitions. We all agree theft is wrong. But taking a few office paper-clips home for personal use “that’s not really stealing, is it?”

The Bottom Line

The bottom line about statistics is this. If you want to know what they really mean, you pretty much have to read the actual research. While there are people legitimately doing their best to put select good questions and produce good summaries, there’s an awful lot of room to spin it any way you want to.

On the data we’ve looked at here I could have written a summary, just as legitimate as the one produced by Theos, to say “Most people disagree with the Pope” arguing that “although people will agree with vague general statements the pope makes, any uniquely papal statements in the sample were largely disagreed with. Clearly the people of Britain tend more towards secularism and utilitarianism”.

So, to return to the quote we started with, it’s quite often a case of “Lies, Damned Lies, and the interpretation of Statistics”. Although there are other ways to fake the statistics (but they too can be at least partly detected by asking for/looking at raw data). As a bonus to close with, at the end of this long post, here’s an example of a different way to manipulate your statistics.

4 Responses to “Lies, Damned Lies, And…”

  1. babylonlurker Says:

    I always think of a fourth kind of lies : Politician’s promises…

    Nicely summary on the selective interpretation of statistics

  2. Jarsto Says:

    @babylonlurker

    Thanks for the compliment. I’ve been getting ever more annoyed with these sorts of misrepresentation for a while, and this dataset was just too good of an example to pass up.

  3. babylonlurker Says:

    Yes, and the clip is, of course, a *classic* example …

  4. Jarsto Says:

    Yes, one of the many (slightly) hidden truths in Yes Minister and Yes Prime Minister.

Leave a Reply