Illustration of density estimation

Here is an illustration of density estimation. We consider records of the incidence of diabetes. The following is quoted verbatim from the data set description:

A population of women who were at least 21 years old, of Pima Indian heritage and living near Phoenix, Arizona, was tested for diabetes according to World Health Organization criteria. The data were collected by the US National Institute of Diabetes and Digestive and Kidney Diseases. We used the 532 complete records after dropping the (mainly missing) data on serum insulin.

In this example, we will construct three density estimates for "glu" (plasma glucose concentration), one conditional on the presence of diabetes, the second conditional on the absence of diabetes, and the third not conditional on diabetes. The conditional density estimates will then be used to construct the probability of diabetes conditional on "glu".

The "glu" data are the following. These were obtained from the MASS package of the R programming language as data(Pima.tr) (200 records) and data(Pima.te) (332 records). Within R, ?Pima.tr and ?Pima.te give a fuller account of the data.

For the records of cases with diabetes (177 records), these are the observed values of "glu", measured in milligrams per deciliter:

195, 97, 128, 137, 189, 92, 143, 149, 164, 140, 121, 105, 176, 171, 199, 154, 167, 184, 139, 134, 131, 158, 112, 181, 168, 144, 107, 125, 125, 115, 150, 140, 148, 117, 80, 124, 103, 124, 112, 148, 145, 151, 144, 187, 129, 167, 180, 177, 152, 198, 188, 168, 197, 158, 130, 151, 115, 194, 184, 95, 100, 138, 100, 175, 133, 128, 129, 155, 148, 78, 197, 166, 118, 119, 102, 90, 111, 171, 180, 109, 100, 136, 122, 160, 162, 88, 117, 173, 170, 156, 152, 163, 104, 179, 129, 128, 109, 109, 196, 109, 85, 162, 134, 181, 179, 119, 184, 113, 155, 101, 106, 119, 107, 146, 144, 161, 128, 124, 155, 109, 152, 122, 102, 125, 196, 189, 173, 116, 105, 193, 136, 172, 173, 144, 129, 151, 181, 95, 189, 180, 104, 158, 135, 125, 84, 163, 145, 128, 90, 186, 187, 176, 111, 181, 174, 138, 112, 97, 179, 136, 155, 145, 111, 162, 142, 169, 93, 129, 187, 173, 174, 120, 147, 187, 181, 128, 170

For the records of cases without diabetes (355 records), these are the observed values of "glu", measured in milligrams per deciliter:

86, 77, 165, 107, 83, 193, 142, 154, 86, 99, 109, 139, 99, 100, 83, 101, 87, 99, 108, 110, 79, 148, 158, 145, 79, 71, 102, 119, 97, 129, 97, 86, 125, 123, 92, 116, 83, 114, 106, 127, 124, 109, 123, 96, 129, 92, 109, 106, 135, 121, 101, 96, 121, 100, 154, 122, 114, 114, 115, 130, 79, 112, 91, 100, 110, 94, 84, 61, 99, 154, 103, 111, 143, 81, 189, 116, 71, 137, 136, 93, 107, 97, 112, 99, 109, 120, 179, 80, 105, 191, 95, 99, 137, 97, 100, 122, 90, 120, 154, 56, 124, 85, 88, 139, 142, 126, 100, 164, 95, 122, 85, 144, 111, 107, 105, 124, 111, 137, 57, 157, 95, 140, 117, 123, 74, 119, 155, 112, 140, 141, 106, 118, 85, 89, 103, 126, 97, 109, 88, 122, 103, 180, 106, 71, 103, 101, 88, 150, 73, 105, 99, 95, 146, 129, 95, 112, 113, 83, 101, 106, 100, 123, 81, 92, 93, 81, 126, 144, 89, 97, 107, 84, 100, 93, 106, 108, 106, 90, 153, 88, 151, 102, 114, 75, 113, 108, 111, 81, 147, 125, 142, 100, 87, 197, 117, 74, 91, 91, 146, 165, 124, 111, 90, 111, 95, 96, 128, 108, 100, 104, 108, 133, 136, 96, 78, 151, 126, 120, 113, 115, 112, 157, 105, 118, 87, 95, 130, 95, 126, 139, 99, 103, 147, 99, 81, 84, 98, 87, 93, 105, 90, 125, 119, 100, 131, 127, 96, 72, 102, 112, 143, 119, 94, 102, 89, 80, 90, 117, 120, 82, 91, 134, 120, 74, 88, 124, 97, 144, 137, 132, 123, 84, 139, 173, 83, 89, 99, 81, 154, 117, 84, 94, 96, 99, 129, 68, 87, 122, 77, 127, 84, 88, 131, 116, 84, 88, 84, 103, 99, 99, 111, 98, 143, 119, 108, 112, 82, 123, 89, 108, 124, 92, 152, 105, 68, 94, 90, 94, 102, 128, 100, 103, 117, 101, 112, 98, 165, 68, 123, 95, 129, 107, 80, 127, 126, 134, 94, 108, 117, 116, 141, 106, 126, 65, 99, 102, 109, 153, 100, 121, 108, 88, 101, 121, 93

The mean of "glu" in the diabetes cases is 143.1 and the standard deviation is 31.26. The mean of "glu" in the non-diabetes cases is 110.0 and the standard deviation is 24.29. From this we see that, in this data set, diabetes cases are associated with greater levels of "glu". This will be made clearer by plots of the estimated density functions.

The first figure shows density estimates of p(glu | diabetes=1), p(glu | diabetes=0), and p(glu). The density estimates are kernel density estimates using a Gaussian kernel. That is, a Gaussian density function is placed at each data point, and the sum of the density functions is computed over the range of the data.

From the density of "glu" conditional on diabetes, we can obtain the probability of diabetes conditional on "glu" via Bayes' rule. For brevity, "diabetes" is abbreviated "db." in this formula.

<math> p(\mbox{diabetes}=1|\mbox{glu})

= \frac{p(\mbox{glu}|\mbox{db.}=1)\,p(\mbox{db.}=1)}{p(\mbox{glu}|\mbox{db.}=1)\,p(\mbox{db.}=1) + p(\mbox{glu}|\mbox{db.}=0)\,p(\mbox{db.}=0)}

<math>

The second figure shows the estimated posterior probability p(diabetes=1 | glu). From these data, it appears that an increased level of "glu" is associated with diabetes.

The incidence of diabetes is a topic of great importance, and this simple analysis can only begin to suggest avenues of further inquiry.

References

• J.W. Smith, J.E. Everhart, W.C. Dickson, W.C. Knowler, and R.S. Johannes. "Using the ADAP learning algorithm to forecast the onset of diabetes mellitus". In Proceedings of the Symposium on Computer Applications in Medical Care (Washington, 1988), ed. R.A. Greenes, pp. 261-265. Los Alamitos, CA: IEEE Computer Society Press, 1988.

• Brian D. Ripley. Pattern Recognition and Neural Networks. Cambridge: Cambridge University Press, 1996.

External links

• UCI Machine Learning Repository Content Summary (http://www.ics.uci.edu/~mlearn/MLSummary.html) (See "Pima Indians Diabetes Database" for the original data set of 732 records, and additional notes.)

Script

The follow commands of the R programming language will create the figures shown above. These commands can be entered at the command prompt by using cut and paste.

library (MASS)
data(Pima.tr)

data(Pima.te)

Pima <- rbind (Pima.tr, Pima.te)
glu <- Pima[,'glu']

d0 <- Pima[,'type'] == 'No'
d1 <- Pima[,'type'] == 'Yes'
base.rate.d1 <- sum(d1)/(sum(d1) + sum(d0))

glu.density <- density (glu)
glu.d0.density <- density (glu[d0])
glu.d1.density <- density (glu[d1])

approxfun (glu.d0.density$x, glu.d0.density$y) -> glu.d0.f
approxfun (glu.d1.density$x, glu.d1.density$y) -> glu.d1.f

p.d.given.glu <- function (glu, base.rate.d1)
{
   p1 <- glu.d1.f(glu) * base.rate.d1
   p0 <- glu.d0.f(glu) * (1 - base.rate.d1)
   p1/(p0+p1)
}

x <- 1:250
y <- p.d.given.glu (x, base.rate.d1)
plot (x, y, type='l', col='red', xlab='glu', ylab='estimated p(diabetes|glu)')

plot (density(glu[d0]), col='blue', xlab='glu', ylab='estimate p(glu), 
   p(glu|diabetes), p(glu|not diabetes)', main=NA)
lines (density(glu[d1]), col='red')
lines (density(glu))

See also

• Parzen window