• The HP filter is a widely used time‑series smoothing and detrending technique that separates an observed series y_t into a long‑run trend component τ_t and a cyclical (short‑term) component c_t:
y_t = τ_t + c_t.
– The method chooses τ_t to minimize the sum of squared deviations from y_t while penalizing the variability (specifically the second difference) of τ_t. The objective is:
minimize Σ_t (y_t − τ_t)^2 + λ Σ_t [(τ_{t+1} − τ_t) − (τ_t − τ_{t−1})]^2,
where λ (lambda) is the smoothing parameter controlling tradeoff between closeness to the data and smoothness of the trend.
Why use it?
– Common in macroeconomics to extract a business‑cycle component (the “cycle”) and a slow‑moving “trend” (e.g., potential output).
– Simple to implement and interpret.
– Works well as a descriptive filter when the noise is roughly white and analysis is historical.
Standard λ (smoothing) choices
– The choice of λ is critical. Common empirical choices:
• quarterly data: λ = 1600 (the conventional baseline)
• annual data: λ = 6.25 (1600*(1/4)^4)
• monthly data: λ = 129600 (1600*(12/4)^4)
– The general scaling uses frequency ratio to the fourth power (λ ∝ frequency^4) because the penalty is on the second difference.
Practical steps to apply the HP filter
1. Inspect your data
• Confirm sampling frequency (annual, quarterly, monthly).
• Plot series and check for outliers, structural breaks, or missing data.
2. Choose λ
• Start with conventional values above. Consider sensitivity checks with several λ values.
3. Apply the filter
• Use a numerical routine (solves a banded linear system). Example libraries:
• Python (statsmodels): from statsmodels.tsa.filters.hp_filter import hpfilter
cycle, trend = hpfilter(series, lamb=1600)
• R (mFilter or mfilter packages): mFilter::hpfilter(series, freq = 4, type = “lambda”)
• Or implement by solving the minimization directly (matrix form).
4. Inspect results
• Plot original series, estimated trend τ_t, and cycle c_t = y_t − τ_t.
• Check if the trend behaves plausibly (no implausible oscillations or instabilities).
• Examine residuals for serial correlation or nonstationarity.
5. Sensitivity and robustness
• Re-run with different λ to see how results change.
• Compare HP‑filtered cycle to alternatives (see below).
• For inference, do not rely solely on HP results; report sensitivity.
6. Real‑time / endpoint caution
• The two‑sided HP filter uses future and past observations equally; end points are poorly estimated (end‑point bias).
• If you need real‑time trend estimates, use a one‑sided filter or state‑space/Kalman methods that are designed for forecasting.
Key advantages
– Easy to implement, interpretable decomposition.
– Common baseline in empirical macro literature — useful for comparison across studies.
Major criticisms and limitations
– Endpoint problem: trend estimates near sample ends are unreliable because the standard HP filter is symmetric and uses future data points.
– Can produce spurious dynamic relationships and distort the data’s spectral properties (i.e., create cycles that are artifacts of the filter).
– May not correspond to the true data‑generating process and can produce misleading cycles if the underlying process is not compatible with the HP penalty.
– Not designed for forecasting; it is primarily a descriptive smoother.
Important critiques and alternatives
– James Hamilton (2017), NBER Working Paper “Why You Should Never Use the Hodrick–Prescott Filter,” argues the HP filter can give misleading results, has poor statistical foundations for many data processes, and performs especially poorly at sample ends. Hamilton proposes an alternative based on autoregressive forecasting of levels (e.g., regress y_{t+4} on four lagged levels) for trend extraction and recommends using methods designed with frequency‑domain properties in mind.
– Alternatives:
• Baxter–King band‑pass filter (designed to isolate cycles in a specific frequency band)
• Christiano–Fitzgerald band‑pass filter (asymmetric or exact finite-sample variants)
• Hamilton’s autoregression‑based approach (practical and simple)
• Kalman filter / state‑space models (flexible, can be used real‑time)
• Loess/local polynomial smoothing, splines
• Wavelet or spectral methods
• Butterworth or other signal‑processing filters
Suggested workflow and best practices
– Use HP as a descriptive tool, not an oracle. Always:
1. Justify λ choice and show sensitivity analysis.
2. Show plots of level, trend, and cycle.
3. Compare to at least one alternative method (e.g., Hamilton or Baxter–King).
4. Avoid overinterpreting cyclical amplitude near sample endpoints.
5. If doing real‑time policy or forecasts, prefer one‑sided filters or state‑space models.
Example code snippets
– Python (statsmodels):
from statsmodels.tsa.filters.hp_filter import hpfilter
cycle, trend = hpfilter(series, lamb=1600)
– R (mFilter):
library(mFilter)
out <- hpfilter(series, freq = 4, type = "lambda") # or set lambda manually
trend <- out$trend
cycle <- out$cycle
Key references
– Hodrick, R. J., & Prescott, E. C. (1997). Postwar U.S. Business Cycles: An Empirical Investigation. Journal of Money, Credit and Banking.
– Hamilton, J. D. (2017). Why You Should Never Use the Hodrick–Prescott Filter. NBER Working Paper No. 23429.
– Investopedia summary
Bottom line
The HP filter is a simple, commonly used tool for decomposing series into trend and cycle. It is useful as a descriptive device when applied carefully with appropriate λ selection and robustness checks. But it has well‑documented statistical drawbacks (end‑point bias, potential to create spurious cycles), so analysts should treat its output cautiously, compare with alternatives, and avoid using it as a sole basis for policy or forecasting decisions.