The 2024 Research and Policy Conference on the Relevance, Timeliness, and Integrity of Federal Statistics is shaping up to be a major event for statisticians, data scientists, and policy makers alike. Running from October 22-24, 2024, at the College Park Marriott Hotel and Conference Center in Hyattsville, MD, this year’s conference will highlight some of the most pressing issues and advancements in the field of federal statistics.
One of the key presentations will be delivered by David S. Matteson, Director of the National Institute of Statistical Sciences (NISS) and Professor at Cornell University, on Tuesday, October 22, 2024, at 3:45 PM. Matteson will present his innovative research in Session C-3: Data Science Methodologies and Applications. The session, organized by Yang Cheng from the National Agricultural Statistics Service (NASS) and chaired by Linda J. Young, also of NASS, will take place in the Patuxent Room.
Matteson’s talk, titled "Drift vs Shift: Decoupling Trends and Changepoint Analysis," introduces a novel approach to analyzing time series data by decoupling long-term trends, known as "drift", from abrupt changes or "shifts". His research offers a breakthrough solution that blends Bayesian trend filtering with machine learning-based regularization, allowing for a more robust and flexible analysis of time-varying data. In many fields—ranging from finance and economics to environmental science and public health—time series data often contains hidden trends and abrupt changes that are crucial for making accurate predictions and strategic decisions. Matteson’s framework addresses these challenges by combining the strengths of two powerful techniques: Bayesian dynamic linear models (DLMs), known for their flexibility and noise-resilient trend estimation, and penalized likelihood methods, which excel at detecting changepoints.
Talk Title: Drift vs Shift: Decoupling Trends and Changepoint Analysis
Presenter: David S. Matteson, Cornell University and National Institute of Statistical Sciences
Abstract: We introduce a new approach for decoupling trends (drift) and changepoints (shifts) in time series. Our locally adaptive model-based approach for robustly decoupling combines Bayesian trend filtering and machine learning based regularization. An over-parameterized Bayesian dynamic linear model (DLM) is first applied to characterize drift. Then a weighted penalized likelihood estimator is paired with the estimated DLM posterior distribution to identify shifts. We show how Bayesian DLMs specified with so-called shrinkage priors can provide smooth estimates of underlying trends in the presence of complex noise components. However, their inability to shrink exactly to zero inhibits direct changepoint detection. In contrast, penalized likelihood methods are highly effective in locating changepoints. However, they require data with simple patterns in both signal and noise. The proposed decoupling approach combines the strengths of both, i.e.\ the flexibility of Bayesian DLMs with the hard thresholding property of penalized likelihood estimators, to provide changepoint analysis in complex, modern settings. The proposed framework is outlier robust and can identify a variety of changes, including in mean and slope. It is also easily extended for analysis of parameter shifts in time-varying parameter models like dynamic regressions. We illustrate the flexibility and contrast the performance and robustness of our approach with several alternative methods across a wide range of simulations and application examples.
The FCSM Conference is an annual forum that brings together top researchers and statisticians to discuss the latest innovations in federal statistics and data science. This year’s theme, "The Relevance, Timeliness, and Integrity of Federal Statistics," emphasizes the importance of accurate and reliable data in shaping policy and driving decision-making at the highest levels of government.
Attendees of the 2024 Research and Policy Conference can look forward to gaining critical insights from Matteson’s presentation, which promises to redefine how changepoint analysis is performed in time series studies. Whether you are dealing with financial data, climate models, or social science research, Matteson’s cutting-edge methods provide a flexible and powerful new tool for statistical analysis.
Learn more: FCSM Research and Policy Conference (fcsmconf.org)