Hybrid Transformer–Evolutionary Algorithm for Predictive Business Forecasting and Strategic Planning

Abstract

Propose HTEA, a Hybrid Transformer-Evolutionary Algorithm model for predictive business forecasting and strategic scenario planning. The proposed HTEA architecture integrates the ability of the Transformer architecture to learn global temporal dependencies in time-series data using multiple heads of self-attention with a differential-evolution/genetic-algorithm based search technique that enables efficient parameter optimisation. Tested across five heterogeneous real-life datasets, HTEA outperforms the most successful baseline (Transformer only) by 46.5%, producing MAPE score of 3.14%. In terms of scenario analysis, HTEA improves upon the best competitor's results by 11.6 percentage points. The results from ablation studies show that each of the architectural components is effective on its own, and the adaptive fusion layer acts as a crucial integrator. Moreover, the computational complexity of the architecture is reasonable and HTEA is implemented openly (Apache-2.0 license).