Welcome to Scholar Publishing Group

International Journal of Big Data Intelligent Technology, 2026, 7(2); doi: 10.38007/IJBDIT.2026.070205.

Research on AI Driven Big Data Platform Intelligent Operation and Anomaly Detection Technology

Author(s)

Chenghao Shi

Corresponding Author:
Chenghao Shi
Affiliation(s)

Georgia Institue of Technology, Atlanta, GA, 30332, USA

Abstract

Due to the combination of cloud-native architecture, distributed storage, real-time computing and multi-tenant resource scheduling, the operational attributes of big data platforms are high-dimensional, dynamic, strongly coupled and have a low labeling level. The existing operation and maintenance method is a static-threshold, manual-inspection approach that does not meet the requirements of low-latency, high-reliability and interpretable governance. This paper introduces an AI-based intelligent operation and maintenance framework for cloud-native big data platforms and focuses on the technical chain of "intelligent perception - anomaly identification - root cause localization - closed-loop handling". It unifies the modelling of multi-source data, including metrics, logs, traces, configurations and events, and integrates strong standardization, time-series predictive reconstruction, dynamic graph association, and adaptive thresholds to achieve anomaly detection. According to research, the core of intelligent operation and maintenance is not a high-accuracy single-point model, but rather the cooperation of data governance, real-time inference, alarm suppression and root cause analysis. This paper also proposes a multi-layered platform architecture and practical implementation plans to offer methodological support for transforming big data platforms from passive fault handling to proactive risk warning.

Keywords

Big Data Platform; Intelligent Operation and Maintenance; Anomaly Detection; Cloud-Native; Time Series Analysis; Root Cause Localization

Cite This Paper

Chenghao Shi. Research on AI Driven Big Data Platform Intelligent Operation and Anomaly Detection Technology. International Journal of Big Data Intelligent Technology (2026), Vol. 7, Issue 2: 40-50. https://doi.org/10.38007/IJBDIT.2026.070205.

References

[1] Zhang, L.; Jia, T.; Jia, M.; Wu, Y.; Liu, A.; Yang, Y.; Wu, Z.; Hu, Zamanzadeh Darban, Z.; Webb, GI; Pan, S.; Aggarwal, CC; Salehi, M. Deep Learning for Time Series Anomaly Detection: A Survey. ACM Computing Surveys, 2024, 57(1): Article 15. DOI: 10.1145/3691338.

[2] Guo, X. (2025, April). Research on Financial Trading Algorithms Based on Deep Reinforcement Learning. In 2025 IEEE 3rd International Conference on Control, Electronics and Computer Technology (ICCECT) (pp. 1711-1715). IEEE.

[3] Guo, X. (2025, March). Research on Blockchain-Based Financial AI Algorithm Integration Methods and Systems. In 2025 IEEE International Conference on Electronics, Energy Systems and Power Engineering (EESPE) (pp. 816-821). IEEE.

[4] Yuan, Y. (2026). Research on Memory Management and Dynamic Reasoning Methods for Intelligent Agents Based on Large Language Models.

[5] Zhang, Z. (2026). Research on the Optimization of Payment Risk Dynamic Monitoring Models Driven by High-dimensional Behavioral Features. Advances in Computer and Communication, 7(3).

[6] Xin, H. (2026). Research on Distributed Data Cleaning and Standardized Mapping for Heterogeneous Logistics Data. Advances in Computer and Communication, 7(2).

[7] Su, J. (2026). Research on Machine Learning-based Performance Prediction and High-reliability Software Evolution Mechanisms for Android Communication Systems.

[8] Wang, Z. (2026). Valuation Enhancement Pathways for Resource Stocks Driven by the Growth of Energy Storage Demand. Engineering Advances, 6(3).

[9] Ma, X. (2026). Research on Elastic Scaling and Resource Scheduling Strategies for Distributed Systems Facing Traffic Fluctuations.

[10] Wang, N. (2026). Research on Digital Marketing Conversion Improvement Strategies and Predictive Analysis Based on User Behavior Segmentation. Journal of Humanities, Arts and Social Science, 10(7).

[11] Wang, Z. (2026). Data-driven Pathways for Optimizing Supply Chain Operational Efficiency in Physical Industries. Advances in Computer and Communication, 7(2).

[12] Jin, C. (2026). Research on the Optimization Strategy of Retail Enterprise Product Portfolio Based on Association Rule Analysis. Advances in Computer and Communication, 7(2).

[13] Ma, W. (2026). Reliable Data Infrastructure Supported by Automated Data Pipelines. Engineering Advances, 6(2).

[14] Wang, Z. (2026). Earnings Quality of Resource Enterprises Under Commodity Price Volatility Transmission.

[15] Xiaoyu Gu. (2025) Research on Generative AI Psychological Intervention Models Based on Multimodal Emotion Recognition. Advances in Computer and Communication, 6(5), 304-309.

[16] Li, J. (2026). Analysis of Advertising Creativity Generation and User Response Mode Based on AIGC.

[17] Li, J. (2026). Research on the Path and Efficiency of Empowering Accurate Advertising Delivery with Data Infrastructure.

[18] Zhang, Y. (2026). Research on LLM-Driven Intelligent Architecture Design and Autonomy Mechanism Construction for Cloud-Native Control Planes.

[19] Liu, X. (2026). Research on the Application of Artificial Intelligence in Consumer Privacy Data Protection. Procedia Computer Science, 279, 956-965.

[20] Liu, X. (2026). Construction of Consumer Data Privacy Protection System Based On Blockchain Technology. Procedia Computer Science, 282, 2082-2091.

[21] Zhang, Y. (2026). Exploration of Enterprise Big Data Microservice Architecture Based on Domain-Driven Design (DDD). Procedia Computer Science, 282, 1994-2003.

[22] Chi, C. (2026). Research on Mortgage Asset Cash Flow Simulation and Risk Transmission Mechanisms across Structural Tranches Based on Loan-Level Data.

[23] Wang, Z. (2026). Research on Data Analysis and Quality Prediction of Manufacturing Processes Based on Improved Deep Learning Algorithms. Procedia Computer Science, 281, 1328-1337.

[24] Ma, X. (2026). Distributed Fault Root Cause Localization and Self-Healing Strategy Generation Based on Causal Inference. Procedia Computer Science, 281, 753-760.

[25] Wang, N. (2026). Research on Integrated Analysis Method of Sales and Operations Data for Management Decision Support.

[26] Han, X. (2026). Research on Adaptive Optimization Methods for Multi-Objective Manufacturing Process Parameters in Complex Assembly Processes. Procedia Computer Science, 279, 366-374.

[27] Sun, J. (2026). Automated Feature Engineering and Screening System for Large-Scale Factor Libraries. Procedia Computer Science, 281, 1282-1290.