Key takeaways
– The error term (often ε, e, or u) is the model component that captures all influences on the dependent variable that are not included in the model’s independent variables.
– The error term is conceptually unobservable; the residual (y − ŷ) is the sample-based observable approximation.
– Proper modeling requires checking assumptions about the error term (zero mean, constant variance, independence, uncorrelated with regressors, often normality) and diagnosing violations (heteroskedasticity, autocorrelation, omitted variables).
– Practical remedies include variable transformation, adding relevant variables, using robust standard errors, or switching to time‑series methods (ARIMA/GARCH) for financial data.
What is an error term?
An error term is the part of a statistical model that represents all the unmeasured influences and randomness that cause the observed dependent variable to differ from the model’s deterministic prediction. In a simple linear regression:
Y = α + βX + ε
ε (the error term) collects everything not captured by α + βX: measurement error, omitted variables, random shocks, and model misspecification.
Error term versus residual
– Error term (ε): the true, theoretical deviation of the outcome from the model’s predicted value. It is unobservable because it depends on the unknown population relationship.
– Residual (e or r): the sample estimate of the error term for observation i: e_i = y_i − ŷ_i. Residuals are computable and are used for diagnostics.
Common notation and interpretation
– E[ε] = 0: errors average to zero (no systematic bias).
– Var(ε) = σ^2 (often assumed constant—homoskedasticity).
– Cov(ε, X) = 0: errors uncorrelated with explanatory variables (exogeneity).
– For inference, errors are often assumed i.i.d. and sometimes normally distributed.
Why the error term matters
– It determines how precise and unbiased parameter estimates and hypothesis tests are.
– Violations of error-term assumptions (heteroskedasticity, autocorrelation, correlation with regressors) can bias standard errors, confidence intervals, and lead to incorrect inferences.
– In finance, unexplained movement in a security’s price is captured by the error term—news, sentiment shifts, microstructure effects, etc.
Practical steps for working with error terms (applies to cross‑section and time‑series regressions)
1. Specify and estimate the model
– Choose likely explanatory variables based on theory and prior evidence.
– Fit the model (OLS for linear models, or appropriate time‑series estimator).
2. Compute residuals
– Residual for observation i: e_i = y_i − ŷ_i.
– Save residuals for diagnostic checks and plots.
3. Visual diagnostics
– Residuals vs fitted values: look for patterns. Random scatter suggests good fit; funnel or structure suggests heteroskedasticity or misspecification.
– Residual histogram / Q‑Q plot: assess normality of errors.
– Residuals over time (for time series): look for autocorrelation or structural breaks.
– Leverage/influence plots (Cook’s distance): detect outliers that disproportionately affect estimates.
4. Formal diagnostic tests
– Heteroskedasticity: Breusch–Pagan test, White test. If present, OLS standard errors are inconsistent.
– Autocorrelation (time series): Durbin‑Watson, Ljung‑Box, Breusch‑Godfrey tests.
– Normality: Shapiro–Wilk, Jarque‑Bera (mainly relevant for small samples and inference).
– Specification tests: Ramsey RESET to detect omitted nonlinearity; tests for omitted variables.
5. Remedies when assumptions fail
– Heteroskedasticity:
– Use heteroskedasticity‑robust standard errors (White/sandwich estimators).
– Model the variance explicitly (e.g., weighted least squares, generalized least squares).
– Autocorrelation:
– Use Newey‑West (HAC) standard errors for time series regressions.
– Switch to time‑series models that capture dynamics (ARIMA, ARIMAX, error‑correction models).
– Conditional heteroskedasticity (volatility clustering in finance):
– Use GARCH family models for conditional variance.
– Nonlinearity or omitted variables:
– Add transforms (log, polynomial terms), interaction terms, or additional variables suggested by theory.
– Endogeneity (Cov(ε, X) ≠ 0):
– Use instrumental variables, difference‑in‑differences, or panel techniques where appropriate.
Example: linear regression of stock price on time
Model: Price_t = α + β·t + ε_t
– ε_t captures everything affecting the observed price other than the deterministic time trend (news, trading volume, market sentiment).
– Steps:
1. Fit OLS and save residuals e_t.
2. Plot e_t vs t to see if variance changes over time (heteroskedasticity) or if there’s serial dependence.
3. Test for autocorrelation (Durbin‑Watson) and heteroskedasticity (Breusch–Pagan).
4. If volatility clustering exists, consider modeling returns rather than prices and use GARCH for conditional volatility.
5. If explanatory variables are omitted (e.g., market index, events), include them to reduce unexplained variation.
Practical checklist before reporting results
– Have you plotted residuals and checked for obvious patterns?
– Did you test for heteroskedasticity and autocorrelation?
– Are standard errors adjusted if assumptions are violated (robust/HAC)?
– Did you consider whether variables are endogenous? If so, did you use an appropriate identification strategy?
– For financial/time‑series data, did you check stationarity and consider modeling returns and conditional variance (GARCH) if needed?
– Are influential observations examined and justified (or removed with caution)?
Simple formulas and definitions
– Residual for observation i: e_i = y_i − ŷ_i
– Sample variance of residuals: s^2 = (1/(n − k)) Σ e_i^2 (k = number of estimated parameters)
– Homoskedasticity assumption: Var(ε_i) = σ^2 for all i
– Heteroskedasticity: Var(ε_i) = σ_i^2, varying across i
Further reading and sources
– Investopedia, “Error Term,” Jiaqi Zhou — https://www.investopedia.com/terms/e/errorterm.asp
– Econometrics textbooks for deeper coverage: Jeffrey M. Wooldridge, Introductory Econometrics; William H. Greene, Econometric Analysis.
If you’d like, I can:
– Walk through a short worked example with simulated data showing residual diagnostics and remedial steps (e.g., apply Breusch–Pagan and show robust SEs).
– Provide code snippets in R or Python (statsmodels) to compute residuals, run tests, and fit GARCH models.