Unified Hyperparameter Optimization Pipeline for Transformer Time-Series Forecasting
A model-agnostic pipeline that wraps hyperparameter optimization around transformer-based time-series forecasting models, structured on the Neural Architecture Search loop: collect the primitive hyperparameters, design the search space, run a search algorithm (Optuna/TPE, grid, random, or Bayesian optimization via Ray Tune), and evaluate. Students learn how to separate model-defining from training hyperparameters, identify which ones (d_model, learning rate, batch size) most affect forecasting error and out-of-memory failures, and apply the same tuning workflow to non-transformer models such as Mamba and TimeMixer.
A Unified Hyperparameter Optimization Pipeline for Transformer-Based Time Series Forecasting Models
Transformer-based time-series forecasting (TSF) models achieve strong accuracy but need heavy hyperparameter tuning, and no standard tuning pipeline existed for them. This paper presents a unified, m…