FreqMi

FreqMi: Coupling Frequency-Domain Attention with Mutual Information for Lightweight Multivariate Time Series Forecasting

Overview

FreqMi is a lightweight framework for multivariate time series forecasting that couples a learnable Stationary Wavelet Transform (SWT) with a dual-branch hybrid attention module:

• Frequency-domain attention: SWT decomposition → rFFT → block-diagonal complex projection → inter-frequency energy coupling attention scored by the product of Hermitian norms: score(l, s) = Σ|q(l)||k(s)|. The projection is block-diagonal (no real/imaginary cross-coupling) and the score contains no phase term.
• Mutual information attention: SWT decomposition → linear projection → attention scored by the Pearson correlation coefficient (R² = cos²θ, MI ≈ −½log(1−R²)), which captures shape-dependent statistical relationships among pseudo time steps.

The two branches are fused with a weighted fusion, and training is regularized by a composite loss with a spectral-fidelity term. The framework achieves competitive accuracy on long-term forecasting benchmarks at a fraction of the parameter and memory cost of large Transformer baselines.

Usage

python run.py --is_training 1 --model_id ETTh1 --model FreqMi --data ETTh1 \
    --root_path ./dataset/ETT-small/ --data_path ETTh1.csv \
    --features M --seq_len 96 --pred_len 96 \
    --e_layers 1 --d_model 32 --d_ff 32 \
    --batch_size 256 --lradj TST --use_norm 1 \
    --enc_in 7 --dec_in 7 --c_out 7 --wv db1 --m 3 \
    --des Exp --itr 3 --fix_seed 2025 --patience 3 \
    --learning_rate 0.02 --l1_weight 0.0005 --attn_type freq   # frequency-domain attention

# Or the Pearson/MI branch:
python run.py ... --attn_type mi

The provided scripts reproduce the full benchmark (8 datasets × 4 horizons × 3 seeds):

bash scripts/multivariate_forecasting/ETT/FreqMi_h1.sh
bash scripts/multivariate_forecasting/Weather/FreqMi_weather.sh
# ... etc. (one script per dataset, runs all pred_lens and seeds sequentially)

Key Arguments

Argument	Description
--attn_type	freq (default) / mi
--beta	MI attention mixing weight (learnable by default)
--fix_beta	Fix beta as a constant instead of learnable
--signed_mi	Use signed MI (sign of Pearson r)
--wv / --m	Wavelet family and SWT decomposition levels
--requires_grad	Learnable wavelet filters (default True)
--rec_lambda / --auxi_lambda / --auxi_type	Time/frequency-domain loss weights
--use_swt	SWT decomposition: 1 (default) / 0 (no SWT subbands)

Datasets

Supported: ETTh1, ETTh2, ETTm1, ETTm2 (ETT-small), Weather, ECL (electricity), Traffic, Solar.

Place the raw data files under ./dataset/ following the standard TS-Lib layout:

dataset/
├── ETT-small/ETTh1.csv, ETTh2.csv, ETTm1.csv, ETTm2.csv
├── weather/weather.csv
├── electricity/electricity.csv
├── traffic/traffic.csv
└── solar/solar_AL.txt

Requirements

• Python 3.8+
• PyTorch (tested with 2.x)
• numpy, pandas, scikit-learn, matplotlib
• PyWavelets (pywt)

Install with:

pip install -r requirements.txt

Repository Structure

FreqMI/
├── run.py                          # entry point
├── model/FreqMi.py                 # model definition
├── layers/                         # SWT, complex attention, MI attention
├── data_provider/                  # data loaders
├── experiments/                    # training/testing pipeline
├── utils/                          # metrics, tools, time features
└── scripts/multivariate_forecasting/  # reproduction scripts per dataset

Citation

If you find this repository useful, please consider citing our paper (coming soon).