Segmented Sequence Prediction Using Variable-Order Markov Model Ensemble

作     者：Yan, Weichao Ma, Hao Yang, Zaiyue 

作者机构：Southern Univ Sci & Technol Guangdong Prov Key Lab Human Augmentat & Rehabil R Sch Automat & Intelligent Mfg Shenzhen Key Lab Control Theory & Intelligent Syst Shenzhen 518055 Peoples R China 

出 版 物：《IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING》 (IEEE Trans Knowl Data Eng)

年 卷 期：2025年第37卷第3期

页      面：1425-1438页

核心收录：

学科分类：0808[工学-电气工程] 08[工学] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

基　　金：National Natural Science Foundataion of China General Program Science, Technology and Innovation Commission of Shenzhen Municipality [JCYJ20220530114815035] Shenzhen Key Laboratory of Control Theory and Intelligent Systems [ZDSYS20220330161800001] 

主　　题：Predictive models Hidden Markov models Data models Sports Natural language processing Analytical models Transformers Training Recurrent neural networks Graphical models Variable-order Markov model sequence prediction probabilistic graphical model recurrent neural network 

摘      要：In recent years, sequence prediction, particularly in natural language processing tasks, has made significant progress due to advanced neural network architectures like Transformer and enhanced computing power. However, challenges persist in modeling and analyzing certain types of sequence data, such as human daily activities and competitive ball games. These segmented sequence data are characterized by short length, varying local dependencies, and coarse-grained unit states. These characteristics limit the effectiveness of conventional probabilistic graphical models and attention-based or recurrent neural networks in modeling and analyzing segmented sequence data. To address this gap, we introduce a novel generative model for segmented sequences, employing an ensemble of multiple variable-order Markov models (VOMMs) to flexibly represent state transition dependencies. Our approach integrates probabilistic graphical models with neural networks, surpassing the representation capabilities of single high-order or variable-order Markov models. Compared to end-to-end deep learning models, our method offers improved interpretability and reduces overfitting in short segments. We demonstrate the efficacy of our proposed method in two tasks: predicting tennis shot types and forecasting daily action sequences. These applications highlight the broad applicability of our segmented sequence modeling approach across diverse domains.