In statistics, the jackknife is a resampling technique especially useful for variance and biasestimation. The jackknife pre-dates other common resampling methods such as the bootstrap. The jackknife estimator of a parameter is found by systematically leaving out each observation from a dataset and calculating the estimate and then finding the average of these calculations. Given a sample of size, the jackknife estimate is found by aggregating the estimates of each -sized sub-sample. The jackknife technique was developed by Maurice Quenouille from 1949 and refined in 1956. John Tukey expanded on the technique in 1958 and proposed the name "jackknife" because, like a physical :wikt:jack-knife|jack-knife, it is a :wikt:rough-and-ready|rough-and-ready tool that can improvise a solution for a variety of problems even though specific problems may be more efficiently solved with a purpose-designed tool. The jackknife is a linear approximation of the bootstrap.
Estimation
The jackknife estimate of a parameter can be found by estimating the parameter for each subsample omitting the i-th observation. For example, if the parameter to be estimated is the population mean of x, we compute the mean for each subsample consisting of all but the i-th data point: These n estimates form an estimate of the distribution of the sample statistic if it were computed over a large number of samples. In particular, the mean of this sampling distribution is the average of these n estimates: One can show explicitly that this equals the usual estimate, so the real point emerges for higher moments than the mean. A jackknife estimate of the variance of the estimator can be calculated from the variance of this distribution of :
The jackknife technique can be used to estimate the bias of an estimator calculated over the entire sample. Say is the calculated estimator of the parameter of interest based on all observations. Let where is the estimate of interest based on the sample with the i-th observation removed, and is the average of these "leave-one-out" estimates. The jackknife estimate of the bias of is given by: and the resulting bias-corrected jackknife estimate of is given by: This removes the bias in the special case that the bias is and removes it to in other cases.
