Mathematics and cancer

Today I want to describe a little mathematical curiosity which may have important implications for our judgement and our decisions.

Let's start with a simple numerical example and consider a group of short people and a group of tall people:

heights in the "short" group {1.45m, 1.50m, 1.55m} average 1.50m
heights in the "tall" group {1.90m, 2.00m, 2.10m} average 2.00m

Moving a single person from the "tall" group to the "short" group may cause the average height of both groups to increase. For instance, by moving the 1.90m person to the "short" group we end up with:

heights in the "short" group {1.45, 1.50, 1.55, 1.90} average 1.60m
heights in the "tall" group {2.00, 2.10} average 2.05m

This phenomenon is named after Will Rogers (see the Wikipedia article).

And now a serious application. We test some people for cancer and we divide them into two groups: "cancer" and "cancer-free". Each group has an average life expectancy, which is clearly shorter for the "cancer" group. Suppose now that an improved cancer detection technique becomes available, we re-test everyone, and some of the people in the "cancer-free" group are moved to the "cancer" group. Again, the average life expectancy of both groups increases: in the healthy group because we removed people who are actually sick, and in the "cancer" group because we have introduced people who have cancer in a very early stage.

Importantly, even though the life expectancy of the "cancer" group increases, it does *not* mean that the quality of the treatments has improved!