Forecast Combination and Model Averaging
Combining predictive signals without confusing model weights, forecast scales, and portfolio weights
QM024 · Machine Learning · Intermediate
Core idea. Forecast combination pools information from multiple forecasting models. The combination rule is itself a model choice and must be evaluated under the same information timing as the component forecasts.
Use it for. Ensembles of statistical, machine-learning, or pretrained forecasting models; robustness to model uncertainty; and systems that aggregate model outputs before a portfolio decision.
It does not establish. That equal weights are always optimal, that estimated weights always improve accuracy, or that a 20% model-combination weight means a 20% portfolio allocation.
The Question
Suppose \(M\) forecasting models produce predictions \(\hat y_{1,t},\ldots,\hat y_{M,t}\) for the same target. Which forecast should be used?
One answer is to select a single winner. Another is to combine them:
\[ \hat y_t^{\text{comb}}=\sum_{m=1}^{M} w_{m,t}\hat y_{m,t}, \]
with weights \(w_{m,t}\) that may be fixed or estimated.
Forecast combination has a long history. Bates and Granger showed how combining forecasts can reduce mean-squared forecast error when component errors contain complementary information (Bates and Granger 1969). Later reviews emphasize that simple combinations often perform surprisingly well relative to more heavily estimated schemes, partly because estimated weights introduce their own sampling error and instability (Clemen 1989; Timmermann 2006).
Why It Matters
A model can be useful without being the best standalone predictor. If two models fail in different states or make different cross-sectional ordering mistakes, their combination can improve the decision process.
But combinations introduce new questions:
- Are the component outputs on a common scale?
- Are the weights fixed ex ante or estimated?
- If estimated, what data were used?
- Are negative weights allowed?
- Do weights sum to one?
- Is the combination performed on forecasts, standardized scores, ranks, probabilities, or decisions?
- Does a model-combination weight get confused with a portfolio allocation weight?
These distinctions are central in investment applications.
Intuition
Consider two noisy estimates of the same future return. If their errors are imperfectly correlated, averaging can cancel part of the noise. If both models make nearly identical errors, averaging adds little.
Now suppose one model outputs return forecasts in decimals while another outputs arbitrary scores. A direct arithmetic average is no longer naturally interpretable. The researcher must first define a compatible integration interface, such as standardization or rank aggregation.
The integration rule can matter as much as the list of component models.
The Method
Fixed linear combination
For forecasts measuring the same target in the same units,
\[ \hat y_t^{\text{comb}}=\mathbf w'\hat{\mathbf y}_t, \]
where \(\hat{\mathbf y}_t=(\hat y_{1,t},\ldots,\hat y_{M,t})'\).
A common restriction is
\[ \sum_{m=1}^{M}w_m=1. \]
Nonnegative weights additionally require \(w_m\ge 0\). Neither constraint is universally required; they define the combination class.
The equal-weight combination is
\[ w_m=\frac{1}{M}. \]
It has no estimated weighting parameters and is therefore a useful robustness benchmark.
Error-variance logic
For two unbiased forecasts with errors \(e_1\) and \(e_2\), a linear combination has error variance
\[ \operatorname{Var}(w e_1+(1-w)e_2) = w^2\sigma_1^2+(1-w)^2\sigma_2^2+2w(1-w)\sigma_{12}. \]
If the error variances and covariance were known and stable, an optimal MSE weight could be derived. In practice they are estimated, can change over time, and may be noisy. This is one reason theoretically refined combinations do not automatically dominate simple rules out of sample (Timmermann 2006).
Estimated weights
Weights can be estimated from historical forecast errors, constrained regression, inverse-error rules, stacking, Bayesian model averaging, or other procedures. Once estimated, they become another layer of model selection.
The key timing rule is:
A combination weight used for forecast origin \(t\) may depend only on information and realized forecast errors available by \(t\).
Using the full historical sample to estimate one set of combination weights and then applying those weights retrospectively to earlier dates gives those earlier decisions future information.
Simple Combinations Are Benchmarks, Not Theorems
The forecasting literature documents a recurring empirical phenomenon: simple combinations often work well and can outperform estimated-optimal combinations (Clemen 1989; Timmermann 2006). The plausible explanations include parameter-estimation error, model instability, misspecification, and limited samples.
That does not imply equal weights are universally best. If relative model quality is stable and estimable, estimated weights can improve on equal weighting. A credible study therefore compares simple and estimated schemes under the same out-of-sample protocol rather than assuming either result in advance.
Forecast Scale Must Be Compatible
Direct averaging is most natural when component predictions estimate the same quantity in the same units.
If models produce heterogeneous scores, common alternatives include:
- standardizing each model’s historical forecast distribution using admissible training data;
- transforming outputs into percentiles or ranks;
- mapping each model to a common calibrated target; or
- combining downstream decisions rather than raw forecasts.
Every transformation changes the integration interface and should be treated as part of the model design.
Rank aggregation is different from forecast averaging
Suppose each model ranks \(N\) assets. A weighted rank score might be
\[ s_{i,t}=\sum_{m=1}^{M} w_m r_{i,m,t}, \]
where \(r_{i,m,t}\) is an asset rank or a normalized rank score.
This is a model-integration rule, not an arithmetic forecast in return units. A 20% contribution from one model means 20% of the model-combination rule under the declared normalization. It does not mean 20% of portfolio capital is allocated to that model.
That distinction should be explicit in any portfolio paper or dashboard.
Combination versus Model Selection
Model selection asks which model to use. Combination asks how to pool several models. These can be nested:
- select eligible models;
- estimate or fix combination weights;
- generate combined forecasts or scores; and
- map the combined output to a decision.
If all four stages are tuned on the same historical sample and only the best final result is reported, the search can be heavily optimized to the sample. QM020 — Time-Series Model Selection and Hyperparameter Tuning covers the relevant nested-evaluation logic.
How to Interpret the Result
A combined model can outperform because component errors are complementary, because averaging reduces variance, because the combination changes a decision boundary, or simply because one component dominates during the evaluation period.
Useful diagnostics therefore include:
- standalone forecast loss for every component;
- pairwise error correlation;
- combined forecast loss;
- weight stability over time;
- sensitivity to reasonable alternative weights;
- decision similarity before and after adding a model; and
- downstream portfolio effects when the forecast is used for investment decisions.
These diagnostics help distinguish a durable ensemble effect from a result driven by one fragile weighting choice.
Financial / Economic Example
Suppose four conventional models forecast next-month returns for eleven assets and a pretrained model supplies an additional forecast signal.
A research design might compare:
- the conventional ensemble alone;
- the pretrained model alone;
- the conventional ensemble plus a fixed pretrained-model contribution;
- equal-weight versus estimated model combinations; and
- standardized-forecast versus rank-based integration.
If the component forecasts, portfolio rule, execution clock, and costs are held fixed while only the integration interface changes, differences in portfolio outcomes can be attributed descriptively to that interface. This remains historical model-adoption evidence, not a causal estimate of what would have happened in real-time production unless the entire technology and information set actually existed at each historical date.
Implementation
A direct combination is simple:
combined = forecasts @ weightsThe research discipline is in how weights were obtained and whether the forecast columns are genuinely compatible.
The accompanying example simulates three forecast models with correlated errors. It compares equal weights with weights estimated from an earlier training period and evaluates both on a later holdout. The sample is deliberately synthetic so no empirical superiority claim is implied.
Hands-on Lab
The lab is explicitly a controlled forecast-error experiment: each synthetic forecast equals the target plus a designed error process. This isolates the finite-sample combination problem instead of pretending the synthetic columns were fitted economic models.
From the QM024 folder:
python labs/python/qm024_forecast_combination.py
python labs/python/qm024_forecast_combination.py --window 60 --common-scale 1.0The script prints the training error-correlation matrix and covariance condition number in addition to standalone and combined holdout losses. Shorter estimation windows and more strongly shared errors can make estimated combination weights less stable; equal weighting is a benchmark, not a guaranteed winner.
Common Mistakes
- Estimating combination weights using future errors. This is look-ahead bias.
- Averaging incompatible output scales. A return forecast and an arbitrary score are not automatically commensurate.
- Confusing model weights with portfolio weights. They live at different layers of the decision process.
- Calling equal weighting optimal. It is a robust benchmark, not a universal optimum.
- Assuming estimated weights must improve. Weight estimation adds sampling error.
- Ignoring weight instability. A strong average result can mask extreme historical weight changes.
- Searching a dense weight grid on the evaluation sample. That converts sensitivity analysis into hidden model selection.
- Reporting a combined portfolio result without standalone components. The source of diversification or improvement becomes opaque.
When Not to Use It
Combination may add little when component models are nearly identical, when one model is known to be invalid for the task, or when output scales cannot be placed on a defensible common interface. In small samples, a complex estimated-weight scheme may be less credible than a simple prespecified rule.
Used in SlackQuant Research
- Equity Alpha — benchmark-aware global equity ETF alpha system using point-in-time selection, rank-based decisions, and active-risk evaluation.
Forecast-to-Rank Allocation (F2R) combines heterogeneous forecasting views before they are translated into a common rank-based decision space.
- F2R Model-Adoption Research examines the decision value of combining conventional and pretrained forecasting components.
Public descriptions distinguish model-combination contribution from portfolio capital allocation and do not expose protected live recipe details.