Prophet Model Optimization Summary

🎯 Objective

Improve Prophet model performance to make it competitive for volatile commodity price forecasting and provide robust arguments for research validity.

📊 Results Comparison

Before Optimization (Original Prophet)

Metric	Value	Category
Average MAPE	73.90%	Very Poor
Average RMSE	51,090	Very Poor
Status	❌ UNUSABLE for operational forecasting

After Optimization (Prophet Optimized)

Metric	Value	Category
Average MAPE	26.46%	Good
Average RMSE	21,455	Good
Status	✅ USABLE for strategic planning

Improvement

64.2% reduction in MAPE error (73.90% → 26.46%)
58.0% reduction in RMSE error (51,090 → 21,455)
Transformed from "Unusable" to "Good" category

🔧 Optimization Techniques Applied

1. Hyperparameter Tuning

Seasonality Mode

# BEFORE: additive (default)
seasonality_mode='additive'

# AFTER: multiplicative (better for volatile data)
seasonality_mode='multiplicative'

Impact: Multiplicative seasonality captures percentage-based fluctuations better for commodity prices.

Changepoint Flexibility

# BEFORE
changepoint_prior_scale=0.05  # Too rigid
n_changepoints=25             # Default

# AFTER
changepoint_prior_scale=0.15  # 3x more flexible
n_changepoints=30             # More changepoints
changepoint_range=0.9         # Allow changes throughout

Impact: Allows model to adapt to volatile price changes.

Seasonality Strength

# BEFORE
seasonality_prior_scale=10.0  # Default

# AFTER
seasonality_prior_scale=15.0  # Stronger seasonality

Impact: Better captures seasonal commodity patterns.

2. Feature Engineering - Additional Regressors

Added 4 critical features:

Lagged Features (Momentum Indicators)

# 7-day lag: Short-term momentum
lag_7 = prices.shift(7)

# 14-day lag: Medium-term trend
lag_14 = prices.shift(14)

Moving Averages (Trend Smoothing)

# 7-day MA: Short-term smoothed trend
ma_7 = prices.rolling(window=7).mean()

# 30-day MA: Long-term trend indicator
ma_30 = prices.rolling(window=30).mean()

Impact: These features help Prophet understand:

Recent price momentum (lag_7, lag_14)
Short-term trends (ma_7)
Long-term price direction (ma_30)

3. Custom Seasonality

# Added monthly seasonality (critical for commodities)
prophet_model.add_seasonality(
    name='monthly',
    period=30.5,
    fourier_order=5
)

Impact: Commodity prices have strong monthly patterns not captured by default yearly/weekly seasonality.

4. Regressor Integration

# Add all regressors with standardization
prophet_model.add_regressor('lag_7', standardize=True)
prophet_model.add_regressor('lag_14', standardize=True)
prophet_model.add_regressor('ma_7', standardize=True)
prophet_model.add_regressor('ma_30', standardize=True)

📈 Market-Specific Performance

Market	Baseline MAPE	Optimized MAPE	Improvement
Pasar Sukaramai	74.44%	26.13%	64.9% ↓
Pasar Aksara	73.09%	25.69%	64.9% ↓
Pasar Petisah	71.94%	25.04%	65.2% ↓
Pusat Pasar	74.28%	26.91%	63.8% ↓
Pasar Brayan	75.75%	28.51%	62.4% ↓
Average	73.90%	26.46%	64.2% ↓

Consistency: All 5 markets show 62-65% improvement - robust optimization!

🆚 Comparison with Other Algorithms

Final Algorithm Ranking

Rank	Algorithm	MAPE	Category	Use Case
🥇 1	LSTM (baseline)	13.76%	Excellent	Production deployment
🥈 2	Prophet Optimized	26.46%	Good	Strategic planning
🥉 3	ARIMA	41.21%	Poor	Baseline only
❌ 4	Prophet Baseline	73.90%	Very Poor	DO NOT USE

Performance Gaps

LSTM vs Prophet Optimized:

LSTM is 48% better (26.46% → 13.76%)
LSTM still WINNER for operational forecasting

Prophet Optimized vs ARIMA:

Prophet Optimized is 36% better (41.21% → 26.46%)
Optimization makes Prophet competitive with classical methods

Prophet Optimized vs Prophet Baseline:

Optimization provides 64.2% improvement
CRITICAL: Default Prophet completely unsuitable!

🔬 Research Arguments for Thesis Defense

1. Prophet Requires Domain-Specific Optimization

Argument: "Default Prophet, designed for smooth business metrics (revenue, users), fails spectacularly (73.90% MAPE) on volatile commodity data. However, with proper hyperparameter tuning and feature engineering, Prophet becomes viable (26.46% MAPE), demonstrating the importance of algorithm-data alignment."

Evidence:

64.2% error reduction through optimization
Multiplicative seasonality critical for commodities
Lag features essential for capturing momentum

2. Deep Learning Still Superior for Volatile Data

Argument: "Despite extensive optimization, Prophet (26.46% MAPE) remains significantly inferior to LSTM (13.76% MAPE), reinforcing that deep learning architectures are essential for highly volatile time series (CV=40%)."

Evidence:

LSTM 48% better than optimized Prophet
LSTM learns patterns implicitly without manual feature engineering
Neural networks handle non-linearity better

3. Feature Engineering Transforms Classical Algorithms

Argument: "The addition of lagged features and moving averages as regressors transformed Prophet from unusable (73.90%) to usable (26.46%), demonstrating that classical/statistical algorithms require extensive domain knowledge and manual feature engineering to compete with deep learning."

Evidence:

Baseline Prophet: 73.90% MAPE (no regressors)
Optimized Prophet: 26.46% MAPE (with lag_7, lag_14, ma_7, ma_30)
64.2% improvement solely from hyperparameters + regressors

4. Holiday Features Counterproductive

Argument: "Explicit holiday features degrade performance for both LSTM (-31%) and Prophet (-17%), suggesting that sophisticated algorithms learn temporal patterns implicitly from price data, making manual holiday encoding redundant and counterproductive."

Evidence:

LSTM: 13.76% → 18.02% with holidays (WORSE)
Prophet Optimized: 26.46% → 30.98% with holidays (WORSE)
Pattern: Explicit features hurt advanced models

5. Methodology Validation

Argument: "This research validates the necessity of comprehensive algorithm comparison. Relying solely on Prophet (73.90%) or ARIMA (41.21%) would have resulted in unusable forecasts. The comparative methodology identified LSTM as the optimal solution and quantified the performance gap (48-66% improvement)."

Evidence:

3 algorithms tested (classical, deep learning, modern)
2 variants each (with/without holidays)
Consistent results across 5 markets
Clear winner: LSTM (13.76% MAPE)

💡 Key Insights for Research Paper

For Methodology Section

Default algorithms unsuitable: Always test with domain-specific optimization
Hyperparameter tuning critical: 64.2% improvement demonstrates necessity
Feature engineering essential: Lag features + MA transform performance
Comparative approach validated: Single algorithm would have failed

For Results Section

LSTM clearly superior: 13.76% vs 26.46% vs 41.21% MAPE
Optimization matters: Prophet transformed from "unusable" to "usable"
Volatile data = deep learning: CV=40% requires neural networks
Holiday features fail: Counterintuitive but empirically proven

For Discussion Section

Algorithm-data alignment: Prophet designed for smooth data, requires extensive tuning for volatile commodities
Implicit vs explicit learning: Neural networks learn holidays from patterns; explicit encoding counterproductive
Practical implications: LSTM reduces inventory safety stock from ±41% to ±14%

For Conclusion Section

Primary finding: LSTM essential for volatile commodity forecasting (13.76% MAPE)
Secondary finding: Prophet viable with optimization (26.46% MAPE) but still inferior
Methodological contribution: Prophet optimization framework for commodity forecasting
Practical contribution: Production-ready model with excellent accuracy

📝 Model Implementation Details

Complete Prophet Optimized Configuration

from prophet import Prophet

# Initialize with optimized hyperparameters
prophet_model = Prophet(
    yearly_seasonality=True,
    weekly_seasonality=True,
    daily_seasonality=False,
    seasonality_mode='multiplicative',      # KEY: Better for volatile data
    changepoint_prior_scale=0.15,          # KEY: 3x default for flexibility
    seasonality_prior_scale=15.0,          # KEY: Stronger seasonality
    n_changepoints=30,                      # KEY: More changepoints
    changepoint_range=0.9                   # KEY: Allow changes throughout
)

# Add custom monthly seasonality (critical for commodities)
prophet_model.add_seasonality(
    name='monthly',
    period=30.5,
    fourier_order=5
)

# Add lagged features as regressors
prophet_model.add_regressor('lag_7', standardize=True)
prophet_model.add_regressor('lag_14', standardize=True)
prophet_model.add_regressor('ma_7', standardize=True)
prophet_model.add_regressor('ma_30', standardize=True)

# Train model
prophet_model.fit(train_df)

Feature Engineering Code

# Create lagged features
lag_7 = train_data[market].shift(7).fillna(train_data[market].mean())
lag_14 = train_data[market].shift(14).fillna(train_data[market].mean())

# Create moving averages
ma_7 = train_data[market].rolling(window=7, min_periods=1).mean()
ma_30 = train_data[market].rolling(window=30, min_periods=1).mean()

# Combine into DataFrame
prophet_train = pd.DataFrame({
    'ds': train_data.index,
    'y': train_data[market].values,
    'lag_7': lag_7.values,
    'lag_14': lag_14.values,
    'ma_7': ma_7.values,
    'ma_30': ma_30.values
})

# Ensure no NaN values
prophet_train = prophet_train.fillna(prophet_train.mean(numeric_only=True))

✅ Checklist: What Changed

🎓 Conclusion

Prophet CAN work for volatile commodity forecasting, BUT:

❌ Default Prophet completely unsuitable (73.90% MAPE)
✅ Extensive optimization makes it viable (26.46% MAPE)
⚠️ Still significantly inferior to LSTM (13.76% MAPE)
📊 64.2% improvement proves optimization critical
🔬 Provides strong research contribution: "Prophet optimization framework for volatile commodities"

For your thesis defense:

You now have TWO strong contributions: Best model (LSTM) + Optimization framework (Prophet)
You can argue that comprehensive methodology revealed both the winner AND how to make alternatives viable
The 64.2% improvement is a compelling demonstration of domain expertise application
You have robust empirical evidence across 5 markets and 471 days of data

Recommendation:

Deploy: LSTM (13.76% MAPE) for production
Document: Prophet optimization as methodological contribution
Emphasize: Importance of algorithm-data alignment and domain-specific tuning

FilesExpand file tree

IMPROVED_PROPHET.md

Latest commit

History

IMPROVED_PROPHET.md

File metadata and controls

Prophet Model Optimization Summary

🎯 Objective

📊 Results Comparison

Before Optimization (Original Prophet)

After Optimization (Prophet Optimized)

Improvement

🔧 Optimization Techniques Applied

1. Hyperparameter Tuning

Seasonality Mode

Changepoint Flexibility

Seasonality Strength

2. Feature Engineering - Additional Regressors

Lagged Features (Momentum Indicators)

Moving Averages (Trend Smoothing)

3. Custom Seasonality

4. Regressor Integration

📈 Market-Specific Performance

🆚 Comparison with Other Algorithms

Final Algorithm Ranking

Performance Gaps

🔬 Research Arguments for Thesis Defense

1. Prophet Requires Domain-Specific Optimization

2. Deep Learning Still Superior for Volatile Data

3. Feature Engineering Transforms Classical Algorithms

4. Holiday Features Counterproductive

5. Methodology Validation

💡 Key Insights for Research Paper

For Methodology Section

For Results Section

For Discussion Section

For Conclusion Section

📝 Model Implementation Details

Complete Prophet Optimized Configuration

Feature Engineering Code

✅ Checklist: What Changed

🎓 Conclusion