Welcome to Scholar Publishing Group

International Journal of Big Data Intelligent Technology, 2026, 7(1); doi: 10.38007/IJBDIT.2026.070120.

Elastic Scaling and Stability Assurance Mechanisms for Distributed Systems under High-Throughput Scenarios

Author(s)

Jiahe Chen

Corresponding Author:
Jiahe Chen
Affiliation(s)

Department of Computer Science, Columbia University, New York City 10027, United States

Abstract

The core challenges of high-throughput distributed systems include scaling delay, weak prediction adaptability, and accuracy quality imbalance.This study proposes a dual mechanism collaborative approach of single dimensional prediction (SLMD LightGBM) and multi-dimensional collaboration (PAHP A): single dimensional prediction achieves accuracy improvements of 24.51% and 11.19% in time series prediction tasks by integrating local mean decomposition, LightGBM, and self updating mechanisms, effectively compensating for response delays; The multidimensional collaboration mechanism integrates predicted values with real-time indicators such as load, service quality, and queuing theory models, and avoids resource mismatches through dual allocation verification to ensure decision-making rationality. Experimental verification shows that the optimized system's violation rate (VR) has decreased to 16.3%, which is 8.5% lower than traditional HPA; The 99th percentile delay is strictly controlled within 880 milliseconds; Reduce resource expenditure by 20%, total cost by 25%, and meet performance threshold under high load. Future optimization will focus on three dimensions: refining the smooth adjustment strategy for Pod contraction stage, and suppressing performance fluctuations through dynamic thresholding and gradient algorithms; Validate the effectiveness of load balancing in complex microservice architectures and explore Pareto optimality for multi service resource allocation; By combining distributed computing with improved predictive model algorithms, we aim to break through the performance bottleneck of large-scale clusters and build a high-throughput scenario stability guarantee system that integrates prediction, decision-making, and execution. This will promote the development of elastic scaling technology towards a more intelligent and robust direction.

Keywords

High-throughput scenario; distributed system; elastic scaling; stability assurance; PAHP A multi-dimensional collaborative mechanism

Cite This Paper

Jiahe Chen. Elastic Scaling and Stability Assurance Mechanisms for Distributed Systems under High-Throughput Scenarios. International Journal of Big Data Intelligent Technology (2026), Vol. 7, Issue 1: 170-178. https://doi.org/10.38007/IJBDIT.2026.070120.

References

[1] Klinke H. Cloud Computing[J]. Springer, Cham, 2025.DOI:10.1007/978-3-031-88130-5_17.

[2] Jiang Q, Zhang W, Lin Z, et al. A Technical Overview of Docker Container Security Threats[C]//International Conference on Cyberspace Simulation and Evaluation.Springer, Singapore, 2025.DOI:10.1007/978-981-96-4509-1_27.

[3] Liu Y, Herranz A H, Sundin R C. RoboKube: Establishing a New Foundation for the Cloud Native Evolution in Robotics[J].IEEE, 2024.DOI:10.1109/ICARA60736.2024.10552996.

[4] Liu, X., & Yang, D. (2025, March). LLM Data Strategy: Improving Data Availability and Efficiency. In Doctoral Symposium on Computational Intelligence (pp. 425-437). Singapore: Springer Nature Singapore.

[5] Zhang, Q. (2025, October). Application of Reinforcement Learning in Dynamic Advertising Content Generation. In 2025 2nd International Conference on Software, Systems and Information Technology (SSITCON) (pp. 1-5). IEEE.

[6] Wang Y. Application of Data Completion and Full Lifecycle Cost Optimization Integrating Artificial Intelligence in Supply Chain[J]. 2025.

[7] Sun, J. (2025). Research on Business Data-driven Risk Prediction Methods Based on Machine Learning. Advances in Computer and Communication, 6(4).

[8] Wu, Y. (2025, October). Multi-Level Belief Rule Base Modeling Architecture and Intelligent Optimization Technology for Decision Support Systems. In 2025 2nd International Conference on Software, Systems and Information Technology (SSITCON) (pp. 1-8). IEEE.

[9] Chen, M. (2026). Research on Privacy-Preserving AI Model Training and Validation Methods Based on Federated Learning.

[10] Xu, D. (2026). Analysis of the impact of video infrastructure optimization on large-scale content quality improvement.

[11] Hou, Y. (2026). Research on Heterogeneous Server Upgrade Strategies and Resource Utilization Efficiency Oriented Toward Green Computing Objectives. Advances in Computer and Communication, 7(1).

[12] Huang, J. (2025, September). Performance Evaluation Index System and Engineering Best Practice of Production-Level Time Series Machine Learning System. In 2025 International Conference on Intelligent Communication Networks and Computational Techniques (ICICNCT) (pp. 01-07). IEEE.

[13] Pan, H. (2025, March). Research on Efficient Computing Model of Hartree Fock and Density Functional Theory Based on GPU Acceleration. In Doctoral Symposium on Computational Intelligence (pp. 485-496). Singapore: Springer Nature Singapore.

[14] Zhang, Q. (2026). Security Improvement and Application of Identity and Access Management in Saas Platform.

[15] Yiting Hong. Differentially Private High-Dimensional Business Data Publishing and Analysis Algorithm. International Journal of Business Management and Economics and Trade (2026), Vol. 7, Issue 1: 28-35.

[16] Wu, W. (2025, June). Construction and optimization of intelligent gateway software management platform based on jenkins cluster management under cloud edge integration architecture in industrial internet of things. In International Conference on 6G Communications Networking and Signal Processing (pp. 633-645). Singapore: Springer Nature Singapore.

[17] Wu Y. Optimization of Generative AI Intelligent Interaction System Based on Adversarial Attack Defense and Content Controllable Generation[J]. 2025.

[18] Chen M. Research on Automated Risk Detection Methods in Machine Learning Integrating Privacy Computing[J]. 2025.

[19] Yanchun Wang. (2025) Research on Enhancing ERP System Efficiency Through AI in Cross-border Supply Chain Environments. Advances in Computer and Communication, 6(5), 268-273.

[20] Wu, Y. (2026). Federated Learning-based Algorithm Design for Privacy Preservation in Cross-domain Data Sharing. Engineering Advances, 6(1).

[21] Ding, J. (2025). Exploration of Process Improvement in Automotive Manufacturing Based on Intelligent Production. International Journal of Engineering Advances, 2(2), 17-23.

[22] Zhang, C., Han, J., Zou, Y., Dong, K., Li, Y., Ding, J., & Han, X. (2024, April). Detecting the anomalies in LiDAR pointcloud. In WCX SAE World Congress Experience. SAE Technical Paper.

[23] Huang, J. (2025, August). Research on Multi-Model Fusion Machine Learning Demand Intelligent Forecasting System in Cloud Computing Environment. In 2025 2nd International Conference on Intelligent Algorithms for Computational Intelligence Systems (IACIS) (pp. 1-7). IEEE.