Goodness-of-Fit

Key takeaways
– Goodness-of-fit evaluates how well observed sample data conform to an assumed probability distribution or model. (Source: Investopedia)
– Common tests: Chi-square (categorical counts), Kolmogorov–Smirnov (K–S, continuous distributions, large samples), Anderson–Darling (A–D, like K–S but more sensitive in tails), Shapiro–Wilk (S–W, tests normality for small–moderate samples). (Source: Investopedia)
– Practical workflow: define hypotheses, choose an alpha, pick the appropriate test, check assumptions, compute the statistic (or use software), compare to critical value or p-value, and interpret results.
– Goodness-of-fit tests assess distributional fit; independence tests (chi-square test of independence) assess association between categorical variables. (Source: Investopedia)

Introduction — What is goodness-of-fit and why it’s important
Goodness-of-fit is a family of statistical tests that measure how closely a sample’s distribution of values matches a specified theoretical distribution (for example, normal, uniform, or a multinomial expectation). Analysts use these tests to validate model assumptions (e.g., normal residuals), to check whether a sample is representative of a population, and to avoid misleading inferences in forecasting, hypothesis testing, and model-building. (Source: Investopedia)

Establish an alpha level
Before testing, set a significance level α (commonly 0.05). This determines the threshold for rejecting the null hypothesis H0 (that the sample follows the specified distribution). If the test’s p-value ≤ α, reject H0; otherwise do not reject H0. (Source: Investopedia)

Which test to use — short guide
– Chi-square goodness-of-fit: categorical/count data (bins). Requires sufficient expected counts (common rule: expected count ≥ 5 per cell or combine bins). Not for continuous raw data unless binned. (Source: Investopedia)
– Kolmogorov–Smirnov (one-sample K–S): continuous data; nonparametric; recommended for large samples; compares empirical CDF to theoretical CDF. Sensitive mainly near the center. (Source: Investopedia)
– Anderson–Darling (A–D): like K–S but gives more weight to tail differences—useful in finance (fat tails). (Source: Investopedia)
– Shapiro–Wilk (S–W): tests normality for continuous data; recommended for small to moderate samples (up to ~2,000). Uses Q–Q style logic and is powerful for normality testing. (Source: Investopedia)
– Other tests: Cramér–von Mises, Lilliefors (a K–S variant when parameters are estimated), Jarque–Bera (for normality using skew/kurtosis), etc.

Detailed descriptions, assumptions, and practical steps

1) Chi-square goodness-of-fit (for categorical counts)
– Purpose: Test whether observed counts across k categories match expected counts from a specified distribution.
– Test statistic: χ² = Σ (Oi − Ei)² / Ei, summed over categories i = 1..k.
– Assumptions: independent observations; categories mutually exclusive; adequate expected counts (commonly Ei ≥ 5).
– Degrees of freedom: df = k − 1 − m, where m = number of parameters estimated from data used to compute expected counts (often m = 0).
– Practical steps:
1. Define H0 (data follow specified proportions) and H1 (do not).
2. Choose α.
3. Compute expected counts Ei = N × pi (where pi are the hypothesized probabilities).
4. If any Ei are too small, combine adjacent categories.
5. Compute χ² and corresponding p-value or compare with critical χ²(df, α).
6. Interpret: p ≤ α → reject H0; p > α → do not reject H0.
– Quick example (fair die): Observed rolls: [8,13,9,12,7,11] out of 60. Expected per face = 10. χ² = Σ (Oi−10)²/10 = … → compute p-value → conclude fairness or not.
– Software: R: chisq.test(); Python: scipy.stats.chisquare().

2) Kolmogorov–Smirnov (one-sample K–S)
– Purpose: Compare an empirical distribution function (EDF) to a theoretical CDF (e.g., normal) for continuous data.
– Test statistic D = max|F_empirical(x) − F_theoretical(x)| over x.
– Assumptions: continuous distribution; no parameters estimated from the same data (otherwise use Lilliefors variant).
– Practical steps:
1. Form H0: sample comes from the specified distribution.
2. Choose α.
3. Compute EDF and theoretical CDF; compute D.
4. Compare D to critical value for sample size at chosen α, or compute p-value.
5. Interpret as above.
– Notes: K–S is more sensitive near the center. For large samples (>2000 recommended by some sources), K–S is suitable. (Source: Investopedia)
– Software: R: ks.test(); Python: scipy.stats.kstest().

3) Anderson–Darling (A–D)
– Purpose: Like K–S but weights differences more heavily in distribution tails.
– Test statistic (for one-sample): A² = −N − S, where S = Σ [(2i − 1)/N] [ln F(Yi) + ln(1 − F(YN+1−i))] using ordered sample values Yi and theoretical CDF F. (Formula from Investopedia.)
– Practical steps:
1. Define hypotheses, choose α.
2. Compute A² (many packages provide values).
3. Compare to critical values (often provided for specific distributions) or use p-value approximations.
4. Interpret; because it emphasizes tails, it’s useful in finance when tail behavior matters. (Source: Investopedia)
– Software: R: nortest::ad.test() or stats::ad.test for some distributions; Python: scipy.stats.anderson() (note: scipy’s anderson() returns critical values rather than p-values for specific distributions).

4) Shapiro–Wilk (S–W)
– Purpose: Test normality for a single continuous variable, recommended for sample sizes up to about 2,000.
– Logic: compares ordered sample quantiles to expected quantiles of the normal distribution; QQ-plot is a related visual tool.
– Practical steps:
1. Set H0: data are normally distributed.
2. Choose α.
3. Calculate S–W statistic (software does this).
4. Use p-value: p ≤ α → reject normality.
5. Plot a QQ plot to visualize deviations (linear pattern ≈ normal).
– Software: R: shapiro.test(); Python: scipy.stats.shapiro().

Goodness-of-fit vs. chi-square test of independence
– Goodness-of-fit tests whether observed counts match a single categorical distribution (expected proportions). By contrast, the chi-square test of independence tests whether two categorical variables are independent in a contingency table. Both use the χ² statistic but answer different questions; ensure you select the correct hypothesis and expected-count computation. (Source: Investopedia)

General practical workflow for any goodness-of-fit test
1. Clarify the question: Are you testing for distributional form (normal, uniform, Poisson) or testing model fit for counts?
2. Inspect data visually: histograms, empirical CDF, QQ-plots, boxplots. Visual checks often show where deviations occur (center vs tails).
3. Choose an appropriate test based on data type, sample size, and which parts of the distribution matter (tails vs center).
4. Check assumptions (independence, expected counts, continuous vs categorical, whether parameters were estimated from the sample).
5. Set α. State H0 and H1 explicitly.
6. Compute test statistic and p-value (or compare to critical values).
7. Draw conclusions, including practical significance and limitations.
8. If H0 rejected, consider what model or transformation might better fit (e.g., log-transform, different distribution family), or use robust methods.

Practical tips and common pitfalls
– Don’t rely solely on p-values. Complement tests with visual diagnostics (QQ plots, histograms, ECDF plots).
– For chi-square, combine categories with low expected counts rather than running the test with many small Ei.
– If parameters of the theoretical distribution are estimated from the same data, use test variants that account for that (e.g., Lilliefors for normality).
– For financial data, prefer tests that give weight to tails (A–D) because tail behavior drives risk measures.
– Use software for accurate p-values; asymptotic/approximate critical values are often used in packages.

Example — fairness of a six-sided die (chi-square goodness-of-fit)
1. Observed counts in 60 rolls: face1=8, face2=13, face3=9, face4=12, face5=7, face6=11.
2. H0: die is fair → expected count per face = 60/6 = 10.
3. Compute χ² = Σ (Oi − 10)² / 10 = (8−10)²/10 + … + (11−10)²/10 = (4+9+1+4+9+1)/10 = 28/10 = 2.8.
4. df = 6 − 1 = 5. Critical χ² at α = 0.05 is ≈ 11.07. Since 2.8 0.05, do not reject H0 — no evidence the die is unfair.
(Use software to get exact p-value: p ≈ 0.732.)

Software reminders
– R: chisq.test(), ks.test(), shapiro.test(), nortest::ad.test() (or packages implementing A–D).
– Python (SciPy): scipy.stats.chisquare(), scipy.stats.kstest(), scipy.stats.shapiro(), scipy.stats.anderson().

Bottom line
Goodness-of-fit tests are an essential toolbox to check model and distributional assumptions. Choose the test that matches your data type, sample size, and analytical priorities (e.g., tail sensitivity). Combine formal tests with graphical diagnostics, and always check the test assumptions before drawing conclusions. (Source: Investopedia — Joules Garcia)

Sources
– Investopedia: “Goodness-of-Fit” by Joules Garcia.

…level of significance to decide whether to accept or reject the null hypothesis.

Below I continue and expand the article with additional sections, practical step‑by‑step procedures, worked examples, guidance on software, limitations and best practices, and a concluding summary.

Source for background definitions: Investopedia — Goodness‑of‑Fit .

What Goodness‑of‑Fit Tests Use Alpha For?
– Alpha (α) is the chosen significance level (commonly 0.05 or 0.01). It defines the threshold for deciding whether observed differences between data and a model are plausibly due to chance.
– If the calculated test statistic produces a p‑value ≤ α (or exceeds the critical value for the statistic), you reject the null hypothesis that “the sample comes from the specified distribution.”
– If p > α (or statistic ≤ critical value), you do not reject the null hypothesis (data are consistent with the specified distribution at that α).

Additional Goodness‑of‑Fit Tests (brief)
– Cramer–von Mises test: like K–S but integrates squared differences between empirical and theoretical CDF across the range.
– Lilliefors test: a K–S variant for normality when mean and variance are estimated from the sample.
– G‑test (likelihood ratio): an alternative to chi‑square for categorical counts; sometimes preferred when sample sizes are moderate.
– Bootstrap or permutation calibration: when assumptions of asymptotic tests are questionable, resampling can estimate the distribution of the test statistic.

Practical Checklist Before Running Any Goodness‑of‑Fit Test
1. Identify whether your data are categorical (nominal/ordinal) or continuous.
2. Decide which theoretical distribution you are testing against (e.g., uniform, Poisson, binomial, normal).
3. Choose an appropriate test for the data type and sample size (see recommendations below).
4. Check assumptions (e.g., no small expected cell counts for chi‑square; independence of observations).
5. Select α (commonly 0.05).
6. Optionally visualize the data first (histogram, empirical CDF, Q‑Q plot) to guide test choice.
7. Run the test, compare the statistic/p‑value with α, and report effect size/diagnostics where possible.

Which Test to Use — Quick Guide
– Categorical counts (expected frequencies available, categories mutually exclusive): Chi‑square goodness‑of‑fit or G‑test.
– Continuous distribution (test whether sample follows a specific continuous distribution):
• Small sample (n ≤ ~2000) and testing for normality: Shapiro‑Wilk.
• Large sample: Kolmogorov‑Smirnov (K‑S) or Anderson‑Darling (A‑D). A‑D is more sensitive in tails.
• If parameters are estimated from data: use Lilliefors or adjust tests accordingly.
– If you care about tail behavior (financial data, extreme‑value sensitivity): Anderson‑Darling.

Practical Steps — Chi‑Square Goodness‑of‑Fit (categorical)
1. Define null H0: observed counts follow the specified distribution (with expected proportions p1,…,pk). Alternative H1: they do not.
2. Choose α (e.g., 0.05).
3. Compute expected counts Ei = N × pi for each category i, where N is sample size.
• Rule of thumb: all Ei should be ≥ 5. If not, combine categories or use an exact/simulation test.
4. Compute the chi‑square statistic:
χ^2 = Σ (Oi − Ei)^2 / Ei (sum over categories)
5. Degrees of freedom: df = k − 1 − m, where k is number of categories and m is number of parameters estimated from data (m=0 if distribution parameters are known).
6. Compare χ^2 to the critical value χ^2_(df,1−α) or compute p‑value. If χ^2 > critical (p ≤ α) reject H0.
7. Report test statistic, df, p‑value, and practical conclusion.

Worked Example — Chi‑Square (fair die test)
– Scenario: You roll a die 60 times, observed face counts: [4, 10, 8, 9, 14, 15]. Test if die is fair (expected each face = 10).
1. H0: die is fair (Ei = 10 for each face). α = 0.05. k = 6, m = 0 → df = 5.
2. Compute χ^2:
(4−10)^2/10 = 36/10 = 3.6
(10−10)^2/10 = 0
(8−10)^2/10 = 4/10 = 0.4
(9−10)^2/10 = 1/10 = 0.1
(14−10)^2/10 = 16/10 = 1.6
(15−10)^2/10 = 25/10 = 2.5
Sum χ^2 = 3.6 + 0 + 0.4 + 0.1 + 1.6 + 2.5 = 8.2
3. Critical χ^2 for df=5 at α=0.05 is ≈ 11.07. Since 8.2 0.05), do not reject H0. The data do not provide strong evidence the die is unfair.

Practical Steps — Kolmogorov‑Smirnov (K‑S) Test (continuous)
1. H0: sample comes from specified continuous distribution F(x) (e.g., Normal(μ,σ^2)). Alternative H1: sample does not.
2. Choose α.
3. Compute empirical CDF Fn(x) and theoretical CDF F(x).
4. Compute D = max_x |Fn(x) − F(x)| (the maximum absolute vertical distance).
5. Compare D to critical value for sample size n and chosen α (or compute p‑value). If D > critical, reject H0.
Notes:
– K‑S assumes continuous distributions and is sensitive to center differences. If distribution parameters are estimated from the data, critical values must be adjusted (use Lilliefors or use simulation).
– The test can have reduced power in the tails; consider A‑D if tails matter.

Practical Steps — Anderson‑Darling (A‑D) Test
1. H0: sample follows specified distribution. A‑D places more weight on tail differences.
2. Choose α.
3. Compute the A^2 statistic (formula involves sorted sample values and theoretical CDF values).
4. Compare A^2 to published critical values (depends on distribution and whether parameters are estimated). If A^2 exceeds critical value, reject H0.
5. Because A‑D emphasizes tails, it is commonly used in finance to capture tail risk.

Practical Steps — Shapiro‑Wilk (S‑W) Test (normality)
1. H0: sample is drawn from a normal distribution. Alternative: not normal.
2. Choose α. Recommended for n up to ~2000.
3. Compute the W statistic (based on a linear combination of ordered sample values). Software returns W and p‑value.
4. If p ≤ α, reject H0 (data deviate from normal). If p > α, do not reject H0.
5. Use a Q‑Q plot alongside the test for visual confirmation.

Software Examples (quick)
– Python (SciPy):
• Chi‑square goodness‑of‑fit (one sample categorical): use scipy.stats.chisquare(observed, f_exp=expected)
• K‑S: scipy.stats.kstest(sample, ‘norm’, args=(mu, sigma))
• Anderson‑Darling: scipy.stats.anderson(sample, dist=’norm’) — returns statistic and critical values (note: SciPy’s anderson supports some distributions)
• Shapiro‑Wilk: scipy.stats.shapiro(sample)
– R:
• Chi‑square: chisq.test(observed, p = expected_proportions)
• K‑S: ks.test(sample, “pnorm”, mean=mu, sd=sigma)
• Anderson‑Darling: nortest::ad.test(sample) or ADGofTest::ad.test(…)
• Shapiro‑Wilk: shapiro.test(sample)

Example — K‑S and Shapiro‑Wilk (conceptual)
– Suppose you have daily returns for a stock (n = 300) and want to test normality.
• Plot histogram and Q‑Q plot first.
• Run Shapiro‑Wilk (works for n ≤ 2000): a small p‑value indicates deviation from normality.
• Run Anderson‑Darling to pay extra attention to tails (if AD statistic is large relative to critical values, reject normality).
• K‑S can be performed, but remember if you estimated mean and variance from the data, K‑S critical values are not strictly valid; use Lilliefors or simulation.

Interpreting Results — Practical Points
– “Do not reject H0” is not proof the model is true—only that data do not show sufficient evidence against H0 at chosen α.
– P‑values near α are ambiguous—consider effect size and graphical diagnostics.
– Large samples can make tiny, practically irrelevant deviations statistically significant. Conversely, small samples may have low power to detect meaningful deviations.
– Always pair tests with visual checks (histograms, density plots, empirical CDFs, Q‑Q plots).
– For categorical chi‑square tests, ensure expected counts are large enough; otherwise use exact tests or combine categories.
– If multiple goodness‑of‑fit tests are run, adjust for multiple comparisons where appropriate.

Goodness‑of‑Fit vs Independence Test (clarify)
– Goodness‑of‑fit test: compares one sample’s distribution to a known theoretical distribution (e.g., does this sample look normal?).
– Independence test (chi‑square test for independence): tests whether two categorical variables in a contingency table are independent (e.g., is preference independent of age group?). Although both use chi‑square statistics, their null hypotheses and data structures differ:
• Goodness‑of‑fit: single categorical variable compared to specified proportions.
• Independence: two categorical variables observed jointly; expected counts come from marginal totals under independence.

Limitations and Common Pitfalls
– Misusing tests for wrong data types (e.g., applying chi‑square to continuous data without binning).
– Ignoring expected count rules for chi‑square (can bias results).
– Using K‑S when parameters are estimated without correction (can lead to incorrect critical values).
– Overreliance on p‑values without considering sample size, effect size, and practical significance.
– Not visualizing data—graphs often reveal the nature of misfit more clearly than a single p‑value.

When Goodness‑of‑Fit Tests Are Especially Important
– Model validation in forecasting and econometrics (verify residuals conform to model assumptions).
– Risk management and finance (assessing tail behavior of asset returns).
– Quality control (checking observed defect rates against expected rates).
– Any situation where decisions/predictions depend on distributional assumptions.

Summary and Practical Recommendations
– Goodness‑of‑fit tests quantify how well sample data match a hypothesized distribution. Choose the test based on data type, sample size, and which part of the distribution (center or tails) is most important.
– Start with visualization (histograms, Q‑Q and P‑P plots, empirical CDF) to identify obvious departures from the assumed distribution.
– Use chi‑square for categorical goodness‑of‑fit (watch expected counts), Shapiro‑Wilk for normality in small/medium samples, K‑S for general continuous distributions (large samples), and Anderson‑Darling when tail fit matters.
– Always report the test statistic, degrees of freedom (if applicable), p‑value, α, and a short interpretation in context. Complement tests with visual diagnostics and consider resampling methods when assumptions are in doubt.
– Remember: statistical significance is not the same as practical significance. Use domain knowledge when deciding whether deviations from the model warrant action.

References
– Investopedia: “Goodness‑of‑Fit” (primary background for this article)
– SciPy and R statistical documentation for test implementations (see scipy.stats and R’s stats/nortest packages for test details and usage).

Editor’s note: The following topics are reserved for upcoming updates and will be expanded with detailed examples and datasets.