Log-linear analysis

Log-linear analysis is a technique used in statistics to examine the relationship between more than two categorical variables. The technique is used for both hypothesis testing and model building. In both these uses, models are tested to find the most parsimonious (i.e., least complex) model that best accounts for the variance in the observed frequencies. (A Pearson's chi-square test could be used instead of log-linear analysis, but that technique only allows for two of the variables to be compared at a time. )

Log-linear analysis is a technique used in statistics to examine the relationship between more than two categorical variables. The technique is used for both hypothesis testing and model building. In both these uses, models are tested to find the most parsimonious (i.e., least complex) model that best accounts for the variance in the observed frequencies. (A Pearson's chi-square test could be used instead of log-linear analysis, but that technique only allows for two of the variables to be compared at a time. )

Fitting criterion

Log-linear analysis uses a likelihood ratio statistic \Chi^2 that has an approximate chi-square distribution when the sample size is large: \Chi^2 = 2\sum O_{ij} \ln \frac{O_{ij}}{E_{ij}}, where \ln = natural logarithm; O_{ij} = observed frequency in cell ij (i = row and j = column); E_{ij} = expected frequency in cell ij . \Chi^2 = the deviance for the model.

Assumptions

There are three assumptions in log-linear analysis: 1. The observations are independent and random; 2. Observed frequencies are normally distributed about expected frequencies over repeated samples. This is a good approximation if both (a) the expected frequencies are greater than or equal to 5 for 80% or more of the categories and (b) all expected frequencies are greater than 1. Violations to this assumption result in a large reduction in power. Suggested solutions to this violation are: delete a variable, combine levels of one variable (e.g., put males and females together), or collect more data. 3. The logarithm of the expected value of the response variable is a linear combination of the explanatory variables. This assumption is so fundamental that it is rarely mentioned, but like most linearity assumptions, it is rarely exact and often simply made to obtain a tractable model. Additionally, data should always be categorical. Continuous data can first be converted to categorical data, with some loss of information. With both continuous and categorical data, it would be best to use logistic regression. (Any data that is analysed with log-linear analysis can also be analysed with logistic regression. The technique chosen depends on the research questions.) The log-linear models can be thought of to be on a continuum with the two extremes being the simplest model and the saturated model. The simplest model is the model where all the expected frequencies are equal. This is true when the variables are not related. The saturated model is the model that includes all the model components. This model will always explain the data the best, but it is the least parsimonious as everything is included. In this model, observed frequencies equal expected frequencies, therefore in the likelihood ratio chi-square statistic, the ratio \frac{O_{ij}}{E_{ij}}=1 and \ln(1)=0 . This results in the likelihood ratio chi-square statistic being equal to 0, which is the best model fit. Other possible models are the conditional equiprobability model and the mutual dependence model. Each log-linear model can be represented as a log-linear equation. For example, with the three variables (A, B, C) the saturated model has the following log-linear equation: \ln(F_{ijk})=\lambda + \lambda_i^A + \lambda_j^B +\lambda_k^C + \lambda_{ij}^{AB} + \lambda_{ik}^{AC}+ \lambda_{jk}^{BC} + \lambda_{ijk}^{ABC}, \, where F_{ijk} = expected frequency in cell ijk ; \lambda = the relative weight of each variable. R with the loglm function of the MASS package (see tutorial) IBM SPSS Statistics with the GENLOG procedure (usage)

References

Text derived from https://en.wikipedia.org/w/index.php?oldid=1337612711 under CC BY-SA 4.0; modifications have been made.