Bootstrapping

Bootstrapping alludes to a German legend about a Baron Münchhausen, who was able to lift himself out of a swamp by pulling himself up by his own hair. In later versions he was using his own boot straps to pull himself out of the sea which gave rise to the term bootstrapping.

In computers, this term refers to any process where a simple system activates a more complicated system. It is the problem of starting a certain system without the system already functioning. It seems just as impossible as "pulling oneself up by the bootstraps" which Baron Münchhausen, according to stories, could do. However, solutions, accordingly called bootstrapping, exist; they are processes whereby a complex system emerges by starting simply and, bit by bit, developing more complex capabilities on top of the simpler ones.

Bootstrapping describes different things in several domains.

Contents

Computing

Bootstrapping is generally a longer term for booting, or the process of starting up any computer. "Bootstrap" most commonly refers to the program that actually begins the initialization of the computers operating system, like GRUB, LILO or NTLDR. Modern personal computers have the ability of using their network interface card(NIC) for bootstrapping, on IA-32(x86) and IA-64(Itanium) this method is implemented by PXE and Etherboot.

Bootstrapping can also refer to the development of successively more complex programming environments. The simplest environment will be, perhaps, a very basic text editor (e.g. ed) and an assembler program. Using these tools, one can write a more complex text editor, and a simple compiler for a higher-level language and so on, until one can have a graphical IDE and an extremely high-level programming language.

Compilers for a programming language X are often written in the same programming language X. One may then wonder how the chicken and egg problem of creating the compiler was solved: if one needs a compiler for language X to obtain a compiler for language X, how did the first compiler get written? Possible methods include:

another compiler for X was written in another language Y;
earlier versions of the compiler were written in a subset of X for which there existed some other compiler;
the compiler for X is cross compiled from another architecture where there exists a compiler for X.
writing the compiler in X, then hand-compiling it from source (most likely in a non-optimized way) and running that on the code to get an optimized compiler. This method is used for gcc.

Methods for distributing compilers in source code include providing a portable bytecode version of the compiler, so as to bootstrap the process of compiling the compiler with itself.

Linguistics

Syntactic bootstrapping is the idea that children use syntactic knowledge they have developed to help learn what words mean -- semantics builds on top of syntax.

Biology

The idea of bootstrapping is significant in a number of fields in the biological sciences. The process by which a fertilised ovum develops into an embryo, particularly the way in which the nuclear genome is expressed differently in its various cells as these differentiate, is one example of bootstrapping. The evolution of progressively better adapted organs through natural selection in a lineage of organisms is another. Some biologists, including Graham Cairns-Smith, believe that the origin of life itself may have been a bootstrap process as one or more systems of biological information storage formed the foundation for successor systems that ultimately supplanted them culminating in the emergence of our current DNA-based system. For more details see the articles on embryology, ontogeny and phylogeny and RNA World.

Electronics

The term bootstrap has a number of meanings in electronics. In classical analog designs, a bootstrap circuit is an arrangement of components used to boost the input impedance of a circuit by using a small amount of negative series feedback. This was often necessary in the early days of bipolar transistors, which inherently have quite a low input impedance. The need for such arrangements has largely been alleviated by the use of modern field effect transistor designs, except when ultra-high input impedances are required.

Another meaning is in connection with the booting process of a computer or other complex system, where the underlying electronics must arrange for the orderly startup of the CPU and related electronics components. This is done long before the CPU is in a state where it can begin to execute software. Nowadays the bootstrap is coordinated by special integrated circuits that monitor the raw power supply and provide the relevant signals to enable the CPU and other chips accordingly.

In a typical car alternator, a DC field current is required before the device can produce any output. Once running, a small portion of the output is fed back to keep the field alive — this can be seen as a form of bootstrapping. The initial priming current is usually obtained from the vehicle's battery — if the battery is totally flat the alternator won't produce any output voltage to recharge it even if the engine is started (by push starting for example) since it will be unable to bootstrap itself. However, in practice, a battery will usually have enough residual charge to avoid this.

Statistics

In statistics bootstrapping is a method for estimating the sampling distribution of an estimator by resampling with replacement from the original sample.

Bootstrap technique was invented by Bradley Efron (1979, 1981, 1982) and further developed by Efron and Tibshirani (1993). "Bootstrap" means that one available sample gives rise to many others by resampling (a concept reminiscent of pulling yourself up by your own bootstrap). While the original objective of cross-validation is to verify replicability of results and that of jackknife is to detect outliers, Efron (1981, 1982) developed bootstrap with inferential purposes.

In bootstrap, the original sample can be duplicated as many times as the computing resources allow, and then this expanded sample is treated as a virtual population. Then samples are drawn from this population to verify the estimators. Obviously the "source" for resampling in bootstrap could be much larger than that in the other two. In addition, unlike cross validation and jackknife, the bootstrap employs sampling with replacement. Indeed, sampling with replacement in a bootstrap is more accurate than sampling without replacement in terms of simulating chance. Further, in cross-validation and Jackknife, the n in the sub-sample is smaller than that in the original sample, but in bootstrap every resample has the same number of observations as the original sample. Thus, the bootstrap method has the advantage of modeling the impacts of the actual sample size (Fan & Wang, 1996).

Julian L. Simon (1997): Resampling: The New Statistics (http://www.resample.com/content/text/index.shtml).

Finance

In finance, bootstrapping refers to the procedure used to calculate the zero coupon yield curve, solving for the maturities where no instruments are available. The method uses interpolation to complete the yield curve, using available zero coupon securities with varying maturities.

It may also mean a company acquiring a competitor with a sole reason of temporarily increasing earnings per share.

Resources

Diaconis, P. & Efron, B. (1983). Computer-intensive methods in statistics. Scientific American, May, 116-130.
Edgington, E. S.(1995). Randomization tests. New York: M. Dekker.
Efron, B. (1979). Bootstrap methods: Another look at the jackknife. The Annals of Statistics, 7, 1-26.
Efron, B. (1981). Nonparametric estimates of standard error: The jackknife, the bootstrap and other methods. Biometrika, 68, 589-599.
Efron, B. (1982). The jackknife, the bootstrap, and other resampling plans. Society of Industrial and Applied Mathematics CBMS-NSF Monographs, 38.
Efron, B., & Tibshirani, R. J. (1993). An introduction to the bootstrap. New York: Chapman & Hall.