Simpson's Paradox

Imagine that you are the head of the local health board. Top of your agenda is Grot's Disease. Although often fatal, there is no recognised treatment. What is to be done about it. Many persons swear by Blogg's Panacea, while others let the disease take its natural course. Should you spend health board money on this alternative therapy? You commission a survey, and the final analysis is
Treatment and outcomes of Grot's Disease
NaturalTreat
Live108153
Die123120
Well there are two surprises. First your scientists have given you a clear answer. Second the alternative therapy does help. Obviously for the 120 patients who took it and died anyway it was a failure. However one is interested in the ratio of those who live to those who die. If a patient takes Blogg's Panacea his odds are better than 50:50, since 153 > 120 but without it the odds are worse than 50:50, since 108 < 123. You decide to spend a large amount of public money to make Blogg's rather expensive Panacea available on the National Health.
You are surprised that this decision proves unpopular with your local feminist co-operative. They have checked the figures for women only.
Female patients
Natural Treat
Live 57 32
Die 100 57
They complain that you have fobbed them off with a treatment that only benefits men, and is useless for women. The survival rate for women without treatment is 57/(57+100)=36.3%. The survival rate for women with treatment is 32/(32+57)=36.0%. They are not rabid feminists and do not accuse you of murdering women because of the 0.3% fall in survival rates. Never the less you cannot fall back on the usual politicians' waffle about the benefit being disappointing. On the strict arithmetic Blogg's Panacea is actually harmful.
Unfortunately, instead of blowing over, the trouble gets worse. One would expect that since the treatment does not benefit women the initial favourable figures must come from a large benefit to men. The local men's group have subtracted the women's figures from the totals for both sexes to get the men's figures. You do not have to look too hard to see the problem.
Male patients
Natural Treat
Live 51 121
Die 23 63
Twice 23 is 46 which is less than 51. Untreated, more than twice as many men survive as die. However, twice 63 is 126 which is more than 121. With the treatment, fewer than twice as many survive as die. 121+63=184 men took the treatment. If it had been merely ineffective 130 should have survived.

The treatment is benefical to men and women, yet harmful to men, and harmful to women. This is Simpson's paradox: Any statistical relationship between two variables may be reversed by including additional factors in the analysis.

You stop wasting public money on a quack treatment, and at a humiliating press conference you promise that the health board will never be caught by Simpson's paradox ever again.

As you sit in your office, relaxing, relieved that the incomprehensible awfulness of Simpson's Paradox is now safely in the past, you receive a phone call from the Home Office. The Home Secretary's own son died off Grott's disease. It couldn't really be helped, he had low blood pressure. You can see from the figures that Blogg's Panacea is of limited benefit to men with low blood pressure.
Men, low B.P.
Natural Treat
Live 4 51
Die 6 57
Obviously the outcome for untreated men with low blood pressure is that 40% live and 60% die. Treament helps, but not alot. It does not improve the odds are far as 50:50. But why in gods name did you stop treating men with high blood pressure? The figures there are much more impressive.
Men, high B.P.
Natural Treat
Live 47 70
Die 17 6
There were one third the deaths, even though slightly more patients were treated than went untreated. At first you think the Home Secretary is talking about special cases, of exceptionally high or low blood pressure, but quickly checking the arithmetic you realise that this is not the case. The data for men has simply been divided into data for men with blood pressure less than a threshold and men with blood pressure greater than or equal to the same threshold. Blogg's Panacea is harmful to all men, but benefical to some and a real life saver to all the others. Simpson's paradox has struck again, reversing the statistical relationship between two variables on the inclusion of an additional factor in the analysis.
As you travel home on the train you are horrified to see your photograph in the newspaper under the headlline Health Board gets it wrong again. The Home Office has leaked the story to the press before they even told you. White faced, you ask if you might take a look at your neighbours paper. The Home Office hasn't leaked the story of men's blood pressures. The story is all about womens' ages.
Grott's Disease in young women
Natural Treat
Live 49 25
Die 19 8
You are getting rather experienced at reading these tables. About half as many young women were treated with Blogg's Panacea as did without. In the live row, 25 is a touch over half of 49, while in the die row, 8 is somewhat less than half 19. It is an unimpressive treatment but better than nothing.
Grott's Disease in older women
Natural Treat
Live 8 7
Die 81 49
There is not much hope in this table for the older woman with Grott's Disease. Blogg's Panacea boosts your chance of survival from 1 in 11 to 1 in 8. Not impressive either, but it does help in both cases. Simpson's paradox strikes again. When it says that
Any statistical relationship between two variables may be reversed by including additional factors in the analysis.
it includes relationships that have already been reversed by the inclusion of previous factors
Simpson's Paradox is deeply troubling. We use the phrase in the final analysis as though there really were a final analysis. Simpson's Paradox casts doubt on this. What I need to do next is to go to the library to try to find
Dawid, A.P. 1979. Conditional Independence in Statistical Theory Journal of the Royal Statistical Society A41(1):1-31
and
Simpson, C. 1951. The Interpretation of Interaction in Contingency Tables. Journal of the Royal Statistical Society B13:238-241
I've been started off on this by the book 
     Computation, Causation, and Discovery.
     Edited by Clark Glymour and Gregory F. Cooper
     The MIT Press
     ISBN 0-262-57124-2