Factor analysis
|
|
Factor analysis is a statistical technique that originated in psychometrics. It is used in the social sciences and in marketing, product management, operations research, and other applied sciences that deal with large quantities of data. The objective is to explain the most of the variability among a number of observable random variables in terms of a smaller number of unobservable random variables called factors. The observable random variables are modeled as linear combinations of the factors, plus "error" terms.
| Contents |
The nature of factor analysis seen via an example
This oversimplified example should not be taken to be realistic. Suppose a psychologist proposes a theory that there are two kinds of intelligence, which let us call "verbal intelligence" and "mathematical intelligence". Evidence for the theory is sought in the examination scores of 1000 students in each of 10 different academic fields. If a student is chosen randomly from a large population, then the student's 10 scores are random variables. The psychologist's theory may say that the average score in each of the 10 subjects for students with a particular level of "verbal intelligence" and a particular level of "mathematical intelligence" is a certain number times the level of "verbal intelligence" plus a certain number times the level of "mathematical intelligence", i.e., it is a linear combination of those two "factors". The numbers by which the two "intelligences" are multiplied are posited by the theory to be the same for all students, and are called "factor loadings". For example, the theory may hold that the average student's aptitude in the science of omphalology is
- { 10 × the student's verbal intelligence } + { 6 × the student's mathematical intelligence }.
The numbers 10 and 6 would be the factor loadings associated with the field of omphalology. Other academic subjects would have factor loadings other than 10 and 6. Two students having identical degrees of verbal intelligence and identical degrees of mathematical intelligence would have different aptitudes in omphalology or any other subject because individual aptitudes differ from average aptitudes. That difference is the "error" — an unfortunate misnomer in statistics that means the amount by which an individual differs from what is average (see errors and residuals in statistics).
The observable data that go into factor analysis would be 10 scores of each of the 1000 students, a total of 10,000 numbers. The factor loadings and levels of the two kinds of intelligence of each student must be inferred from the data. Indeed, even the number of factors (two, in this example) must be inferred from the data.
Mathematical model of the same concrete example
In the example above, for i = 1, ..., 1,000 the ith student's scores are
- <math>\begin{matrix}x_{1,i} & = & \mu_1 & + & \ell_{1,1}v_i & + & \ell_{1,2}m_i & + & \varepsilon_{1,i} \\
\vdots & & \vdots & & \vdots & & \vdots & & \vdots \\ x_{10,i} & = & \mu_{10} & + & \ell_{10,1}v_i & + & \ell_{10,2}m_i & + & \varepsilon_{10,i} \end{matrix}<math>
where
- vi is the ith student's "verbal intelligence",
- mi is the ith student's "mathematical intelligence",
- εk,i is the difference between the ith student's score in the kth subject and the average score in the kth subject of all students whose levels of verbal and mathematical intelligence are the same as those of the ith student,
- <math>\ell_{k,j}<math> are the factor loadings for the kth subject, for j = 1, 2.
In matrix notation, we have
- <math>X=\mu+LF+\epsilon<math>
where
- X is a 10 × 1,000 matrix of observable random variables,
- μ is a 10 × 1 column vector of unobservable constants (in this case "constants" are quantities not differing from one individual student to the next; and "random variables" are those assigned to individual students; the randomness arises from the random way in which the students are chosen),
- L is a 10 × 2 matrix of unobservable constants,
- F is a 2 × 1,000 matrix of unobservable random variables,
- ε is a 10 × 1,000 matrix of unobservable random variables.
Now observe that by doubling the scale on which "verbal intelligence"—the first component in each column of F—is measured, and simultaneously halving the factor loadings for verbal intelligence makes no difference to the model. Thus, no generality is lost by assuming that the standard deviation of verbal intelligence is 1. Likewise for mathematical intelligence. Moreover, for similar reasons, no generality is lost by assuming the two factors are uncorrelated with each other. The "errors" ε are taken to be independent of each other. The variances of the "errors" associated with the 10 different subjects are not assumed to be equal.
The values of the loadings L, the averages μ, and the variances of the "errors" ε must be estimated given the observed data X. [How this is done is a subject that must get addressed in this article, which remains "under construction".]
Factor analysis in psychometrics
History
Charles Spearman pioneered the use of factor analysis in the field of psychology and is sometimes credited with the invention of factor analysis. He discovered that schoolchildren's scores on a wide variety of seemingly unrelated subjects were positively correlated, which led him to postulate that a general mental ability, or g, underlies and shapes human cognitive performance. His postulate now enjoys broad support in the field of intelligence research, where it is known as g theory.
Raymond Cattell expanded on Spearman’s idea of a two-factor theory of intelligence after performing his own tests and factor analysis. He used a multi-factor theory to explain intelligence. Cattell’s theory addressed alternate factors in intellectual development, including motivation and psychology. Cattell also developed several mathematical methods for adjusting psychometric graphs, such as his "scree" test and similarity coefficients. His research lead to the development of his theory of fluid and crystallized intelligence. Cattell was a strong advocate of factor analysis and psychometrics. He believed that all theory should be derived from research, which supports the continued use of empirical observation and objective testing to study human intelligence.
Applications in psychology
Factor analysis has been used in the study of human intelligence as a method for comparing the outcomes of (hopefully) objective tests and to construct matrices to define correlations between these outcomes, as well as finding the factors for these results. The field of psychology that measures human intelligence using quantitative testing in this way is known as psychometrics (psycho=mental, metrics=measurement).
Advantages
- Offers a much more objective method of testing intelligence in humans
- Allows for a satisfactory comparison between the results of intelligence tests
- Provides support for theories that would be difficult to prove otherwise
Disadvantages
- "...each orientation is equally acceptable mathematically. But different factorial theories proved to differ as much in terms of the orientations of factorial axes for a given solution as in terms of anything else, so that model fitting did not prove to be useful in distinguishing among theories." (Sternberg, 1977). This means that even though all rotations are mathematically equal, they all come up with different results, and it is impossible to judge the proper rotation.
- "[Raymond Cattell] believed that factor analysis was 'a tool that could be applied to the study of behavior and ... might yield results with an objectivity and reliability rivaling those of the physical sciences (Stills, p. 114).'" [1] (http://www.indiana.edu/~intell/rcattell.shtml) In other words, one’s gathering of data would have to be perfect and unbiased, which will probably never happen.
- Interpreting factor analysis is based on using a “heuristic”, which is a solution that is "convenient even if not absolutely true" (Richard B. Darlington). More than one interpretation can be made of the same data factored the same way.
Factor analysis in marketing
The basic steps are:
- Identify the salient attributes consumers use to evaluate products in this category.
- Use quantitative marketing research techniques (such as surveys) to collect data from a sample of potential customers concerning their ratings of all the product attributes.
- Input the data into a statistical program and run the factor analysis procedure. The computer will yield a set of underlying attributes (or factors).
- Use these factors to construct perceptual maps and other product positioning devices.
Information collection
The data collection stage is usually done by marketing research professionals. Survey questions ask the respondent to rate a product from one to five (or 1 to 7, or 1 to 10) on a range of attributes. Anywhere from five to twenty attributes are chosen. They could include things like: ease of use, weight, accuracy, durability, colourfulness, price, or size. The attributes chosen will vary depending on the product being studied. The same question is asked about all the products in the study. The data for multiple products is coded and input into a statistical program such as SPSS or SAS.
Analysis
The analysis will isolate the underlying factors that explain the data. Factor analysis is an interdependence technique. The complete set of interdependent relationships are examined. There is no specification of either dependent variables, independent variables, or causality. Factor analysis assumes that all the rating data on different attributes can be reduced down to a few important dimensions. This reduction is possible because the attributes are related. The rating given to any one attribute is partially the result of the influence of other attributes. The statistical algorithm deconstructs the rating (called a raw score) into its various components, and reconstructs the partial scores into underlying factor scores. The degree of correlation between the initial raw score and the final factor score is called a factor loading. There are two approaches to factor analysis: "principal component analysis" (the total variance in the data is considered); and "common factor analysis" (the common variance is considered).
Note that there are very important conceptual differences between the two approaches, an important one being that the common factor model involves a testable model whereas principal components does not. This is due to the fact that in the common factor model, unique variables are required to be uncorrelated, whereas residuals in principal components are correlated. Finally, components are not latent variables; they are linear combinations of the input variables, and thus determinate. Factors, on the other hand, are latent variables, which are indeterminate. If your goal is to fit the variances of input variables for the purpose of data reduction, you should carry out principal components analysis. If you want to build a testable model to explain the intercorrelations among input variables, you should carry out a factor analysis.
The use of principle components in a semantic space can vary somewhat because the components may only "predict" but not "map" to the vector space. This produces a statistical principle component use where the most salient words or themes represent the preferred basis.
Advantages
- both objective and subjective attributes can be used
- it is fairly easy to do, inexpensive, and accurate
- it is based on direct inputs from customers
- there is flexibility in naming and using dimensions
Disadvantages
- usefulness depends on the researchers ability to develop a complete and accurate set of product attributes - If important attributes are missed the procedure is valueless.
- naming of the factors can be difficult - multiple attributes can be highly correlated with no apparent reason.
- factor analysis will always produce a pattern between variables, no matter how random.
Bibliography
- Charles Spearman. Retrieved July 22, 2004, from http://www.indiana.edu/~intell/spearman.shtml
- Factor Analysis. (2004). Retrieved July 22, 2004, from http://comp9.psych.cornell.edu/Darlington/factor.htm
- Factor Analysis. Retrieved July 23, 2004, from http://www2.chass.ncsu.edu/garson/pa765/factor.htm
- Raymond Cattell. Retrieved July 22, 2004, from http://www.indiana.edu/~intell/rcattell.shtml
- Sternberg, R.J.(1990). The geographic metaphor. In R.J. Sternberg, Metaphors of mind: Conceptions of the nature of intelligence (pp.85-111). New York: Cambridge.
- Stills, D.L. (Ed.). (1989). International encyclopedia of the social sciences: Biographical supplement (Vol. 18). New York: Macmillan.