PREDICTING STOCK PRICE DIRECTION IN FINANCIAL MARKETS USING MACHINE LEARNING AND HYBRID MODELS

Authors

DOI:

https://doi.org/10.32782/2617-5940.1.2026.20

Keywords:

financial market, stock price prediction, machine learning, hybrid models, investment management, risk management, financial time series

Abstract

In the context of the rapid growth of financial information and the active adoption of artificial intelligence technologies, forecasting financial asset price dynamics is becoming increasingly important for making informed investment decisions under conditions of heightened volatility and potential market shocks. This drives growing interest in the application of machine learning methods capable of identifying complex nonlinear relationships and rapidly adapting to changes in non-stationary time series. The aim of the article is to conduct a cross-sectoral empirical analysis of the predictive performance of econometric models and machine learning algorithms in comparison with classical statistical approaches, as well as to evaluate a hybrid ARIMA–XGBoost model for predicting stock price direction. The object of the study is financial time series of six multinational corporations from different sectors of the economy, taking into account macroeconomic indicators (S&P 500, VIX, NASDAQ) that reflect key information interaction channels in financial markets. To ensure the correctness and reproducibility of the results, an expanding window walk-forward validation approach is applied, which accounts for changes in market conditions over time. The study is based on the use of exponential smoothing models (ETS), ARIMA, GARCH, and gradient boosting algorithms (XGBoost, LightGBM). The proposed two-component hybrid ARIMA–XGBoost model combines time series decomposition with residual learning, ensuring the integration of linear and nonlinear effects. The empirical results indicate that the hybrid approach demonstrates higher predictive performance and greater robustness to market shocks compared to standalone econometric and machine learning models, demonstrating improved stability across different market conditions. The practical value of the results lies in their applicability for developing effective risk management policies, improving portfolio management efficiency, and implementing artificial intelligence tools in the context of financial market digitalization.

References

Chen H.-Y., Jenweeranon P., Alam N. Global Perspectives in the Metaverse. Palgrave Macmillan, Cham. 2024. DOI: https://doi.org/10.1007/978-3-031-54802-4_1

Garg K. Digital identities in the metaverse: Privacy, security, and user authentication in virtual financial systems. International Journal of Financial Engineering. 2024. № 11(4). DOI: https://doi.org/10.1142/S242478632442009X

Mishchenko V., Naumenkova S., Mishchenko S., Тishchenko I. Formation and Functioning of Financial Metaverse Platforms. Financial and Credit Activity Problems of Theory and Practice. 2025. № 1(60). Р. 111–122. DOI: https://doi.org/10.55643/fcaptp.1.60.2025.4689

Міщенко В.І., Науменкова С.В. Механізми державної підтримки використання штучного інтелекту для забезпечення стійкості економічного розвитку. Економіка України. 2024. № 5(750). С. 30–56. DOI: https://doi.org/10.15407/economyukr.2024.05.030

Financial Stability Board. The Financial Stability Implications of Artificial Intelligence. 2024. URL: https://www.fsb.org/2024/11/the-financial-stability-implications-of-artificial-intelligence/ (дата звернення: 10.03.2026)

Fabozzi F. J. The Handbook of Financial Instruments. John Wiley & Sons, Inc. 2018. 864 p.

Sanders A., Cornett M. Financial Markets and Institution. The McGraw-Hill, 2012. 754 р.

Fama E., French K. International Tests of a Five-Factor Asset Pricing Model. Journal of Financial Economics. 2017. № 123(3). Р. 441–463. DOI: https://doi.org/10.1016/j.jfineco.2016.11.004

Paliienko O., Naumenkova S., Mishchenko, S. An empirical investigation of the Fama-French five-factor model. Investment Management and Financial Innovations. 2020. № 17(1). P. 143–155. DOI: https://doi.org/10.21511/imfi.17(1).2020.13

Hyndman R. J., Athanasopoulos G. Forecasting: Principles and Practice. 2021. URL: https://otexts.com/fpp3/ (дата звернення: 15.01.2026)

Swamy S. R., Rajgoli S. R., Hegde T. Stock Market Prediction with Machine Learning: A Comprehensive Review. Indiana Journal of Multidisciplinary Research. 2024. № 4(3). P. 265–271. DOI: https://doi.org/10.5281/zenodo.12685839

Guo Z. Stock Market Price Prediction Using Machine Learning Models. 2nd International Conference on Financial Technology and Business Analysis. 2023. № 45. P. 102–111. DOI: https://doi.org/10.54254/2754-1169/45/20230266

Kayit A., Ismail M. Advancing stock price prediction through the development of hybrid ensembles: a comprehensive comparative analysis of machine learning approaches. Journal of Big Data. 2025. № 12. 232. DOI: https://doi.org/10.1186/s40537-025-01185-8

Nti I., Adekoya F., Weyori B. A comprehensive evaluation of ensemble learning for stock market prediction. Journal of Big Data. 2020. № 7. 20. URL: https://link.springer.com/article/10.1186/s40537-020-00299-5

Tan Z., Yan Z., Zhu G. Stock selection with random forest: An exploitation of excess return in the Chinese stock market. Heliyon. 2019. № 5(8). DOI: https://doi.org/10.1016/j.heliyon.2019.e02310

Chen Q. Stock price forecasting using machine-learning methods. Applied and Computational Engineering. 2024. № 52. Р. 208–214. DOI: https://doi.org/10.54254/2755-2721/52/20241570

Xie Y. Stock Price Forecasting: Traditional Statistical Methods and Deep Learning Methods. Highlights in Business, Economics and Management. 2023. № 21. P. 740–745. DOI: https://doi.org/10.54097/hbem.v21i.14754

Chen T., Guestrin C. XGBoost: A Scalable Tree Boosting System. The 22nd ACM SIGKDD International Conference. 2016. Р. 785–794. DOI: https://doi.org/10.1145/2939672.2939785

Mutinda J. K., Langat A. K. Stock price prediction using combined GARCH-AI models. Scientific African. 2024. № 26. P. 1–14. DOI: https://doi.org/10.1016/j.sciaf.2024.e02374

Han H., Liu Z., Barrios M. et al. Time series forecasting model for non-stationary series pattern extraction using deep learning and GARCH modeling. Journal of Cloud Computing. 2024. № 13. 2. DOI: https://doi.org/10.1186/s13677-023-00576-7

Li M, Khan D, Alshanbari H, El-Bagoury A. Prediction of complex stock market data using an improved hybrid EMD-LSTM model. Applied Sciences. 2023. № 13(3). 1429. DOI: https://doi.org/10.3390/app13031429.

Inani S., Pradhan H., Kumar S., Biswas B. Navigating the technical analysis in stock markets: Insights from bibliometric and topic modeling approaches. Investment Management and Financial Innovations. 2024. № 21(1). Р. 275–288. DOI: https://doi.org/10.21511/imfi.21(1).2024.21

Міщенко С.В., Науменкова С.В., Тіщенко Є.О. Синергія штучного інтелекту та блокчейн у фінансовому секторі. Вісник Київського національного університету імені Тараса Шевченка. Економіка. 2025. № 2(227). С. 54–64. DOI: https://doi.org/10.17721/1728-2667.2025/227-2/7

Yahoo Finance. URL: https://finance.yahoo.com/ (дата звернення: 20.01.2026)

Chen H.-Y., Jenweeranon P., Alam N. (2024) Global Perspectives in the Metaverse. Palgrave Macmillan, Cham. DOI: https://doi.org/10.1007/978-3-031-54802-4_1 DOI: https://doi.org/10.1007/978-3-031-54802-4_1

Garg K. (2024) Digital identities in the metaverse: Privacy, security, and user authentication in virtual financial systems. International Journal of Financial Engineering, vol. 11(4). DOI: https://doi.org/10.1142/S242478632442009X DOI: https://doi.org/10.1142/S242478632442009X

Mishchenko V., Naumenkova S., Mishchenko S., Тishchenko I. (2025) Formation and Functioning of Financial Metaverse Platforms. Financial and Credit Activity Problems of Theory and Practice, vol. 1(60), pp. 111–122. DOI: https://doi.org/10.55643/fcaptp.1.60.2025.4689 DOI: https://doi.org/10.55643/fcaptp.1.60.2025.4689

Mishchenko V., Naumenkova S. (2024) Mekhanizmy derzhavnoi pidtrymky vykorystannia shtuchnoho intelektu dlia zabezpechennia stiikosti ekonomichnoho rozvytku [State Support Mechanisms for the Use of Artificial Intelligence to Ensure Resilience of Economic Development]. Economy of Ukraine, vol. 67(5), pp. 30–56. DOI: https://doi.org/10.15407/economyukr.2024.05.030 DOI: https://doi.org/10.15407/economyukr.2024.05.030

Financial Stability Board. (2024) The Financial Stability Implications of Artificial Intelligence. Available at: https://www.fsb.org/2024/11/the-financial-stability-implications-of-artificial-intelligence/ (accessed March 10, 2026)

Fabozzi F. J. (2018) The Handbook of Financial Instruments. John Wiley & Sons, Inc. 864 p.

Sanders A., Cornett M. (2012) Financial Markets and Institution. The McGraw-Hill. 754 р.

Fama E., French K. (2017) International Tests of a Five-Factor Asset Pricing Model. Journal of Financial Economics, vol. 123(3), pp. 441–463. DOI: https://doi.org/10.1016/j.jfineco.2016.11.004 DOI: https://doi.org/10.1016/j.jfineco.2016.11.004

Paliienko O., Naumenkova S., Mishchenko, S. (2020) An empirical investigation of the Fama-French five-factor model. Investment Management and Financial Innovations, vol. 17(1), pp. 143–155. DOI: https://doi.org/10.21511/imfi.17(1).2020.13 DOI: https://doi.org/10.21511/imfi.17(1).2020.13

Hyndman R. J., Athanasopoulos G. (2021) Forecasting: Principles and Practice. Monash University, Australia. Available at: https://otexts.com/fpp3/ (accessed January 15, 2026)

Swamy S. R., Rajgoli S. R., Hegde T. (2024) Stock Market Prediction with Machine Learning: A Comprehensive Review. Indiana Journal of Multidisciplinary Research, vol. 4(3), pp. 265–271. DOI: https://doi.org/10.5281/zenodo.12685839

Guo, Z. (2023) Stock Market Price Prediction Using Machine Learning Models. 2nd International Conference on Financial Technology and Business Analysis, no. (45), pp. 102–111. DOI: https://doi.org/10.54254/2754-1169/45/20230266 DOI: https://doi.org/10.54254/2754-1169/45/20230266

Kayit A., Ismail M. (2025). Advancing stock price prediction through the development of hybrid ensembles: a comprehensive comparative analysis of machine learning approaches. Journal of Big Data, vol. 12. 232. DOI: https://doi.org/10.1186/s40537-025-01185-8 DOI: https://doi.org/10.1186/s40537-025-01185-8

Nti I., Adekoya F., Weyori B. (2020). A comprehensive evaluation of ensemble learning for stock market prediction. Journal of Big Data, vol. 7. 20. DOI: https://doi.org/10.1186/s40537-020-00299-5 DOI: https://doi.org/10.1186/s40537-020-00299-5

Tan Z., Yan Z., Zhu G. (2019) Stock selection with random forest: An exploitation of excess return in the Chinese stock market. Heliyon, vol. 5(8). DOI: https://doi.org/10.1016/j.heliyon.2019.e02310 DOI: https://doi.org/10.1016/j.heliyon.2019.e02310

Chen Q. (2024) Stock price forecasting using machine-learning methods. Applied and Computational Engineering, vol. 52, pp. 208–214. DOI: https://doi.org/10.54254/2755-2721/52/20241570 DOI: https://doi.org/10.54254/2755-2721/52/20241570

Xie Y. (2023) Stock Price Forecasting: Traditional Statistical Methods and Deep Learning Methods. Highlights in Business, Economics and Management, vol. 21, pp .740–745. DOI: https://doi.org/10.54097/hbem.v21i.14754 DOI: https://doi.org/10.54097/hbem.v21i.14754

Chen T., Guestrin, C. (2016) XGBoost: A Scalable Tree Boosting System. The 22nd ACM SIGKDD International Conference, pp. 785–794. DOI: https://doi.org/10.1145/2939672.2939785 DOI: https://doi.org/10.1145/2939672.2939785

Mutinda J. K., & Langat A. K. (2024) Stock price prediction using combined GARCH-AI models. Scientific African, vol. 26, pp. 1–14. DOI: https://doi.org/10.1016/j.sciaf.2024.e02374 DOI: https://doi.org/10.1016/j.sciaf.2024.e02374

Han, H., Liu, Z., Barrios Barrios, M. et al. (2024). Time series forecasting model for non-stationary series pattern extraction using deep learning and GARCH modeling. Journal of Cloud Computing, vol. 13. 2. DOI: https://doi.org/10.1186/s13677-023-00576-7 DOI: https://doi.org/10.1186/s13677-023-00576-7

Li M, Khan D, Alshanbari H, El-Bagoury A. (2023). Prediction of complex stock market data using an improved hybrid EMD-LSTM model. Applied Sciences, vol. 13(3), 1429. DOI: https://doi.org/10.3390/app13031429. DOI: https://doi.org/10.3390/app13031429

Inani S., Pradhan H., Kumar S., Biswas B. (2024) Navigating the technical analysis in stock markets: Insights from bibliometric and topic modeling approaches. Investment Management and Financial Innovations, vol. 21(1), pp. 275–288. DOI: https://doi.org/10.21511/imfi.21(1).2024.21 DOI: https://doi.org/10.21511/imfi.21(1).2024.21

Mishchenko S., Naumenkova S., Tishchenko І. (2025) Synerhiia shtuchnoho intelektu ta blokchein u finansovomu sektori [The Synergy of Artificial Intelligence and Blockchain Technologies sn the Financial Sector]. Bulletin of Taras Shevchenko National University of Kyiv. Economic, vol. 2(227), pp. 54–64. DOI: https://doi.org/10.17721/1728-2667.2025/227-2/7 DOI: https://doi.org/10.17721/1728-2667.2025/227-2/7

Yahoo Finance. Available at: https://finance.yahoo.com/ (accessed January 20, 2026).

Published

2026-06-24

How to Cite

Naumenkova, S., Mishchenko, S., & Yarych, N. (2026). PREDICTING STOCK PRICE DIRECTION IN FINANCIAL MARKETS USING MACHINE LEARNING AND HYBRID MODELS. Collection of Scientific Papers of the State Tax University, (1), 129–137. https://doi.org/10.32782/2617-5940.1.2026.20