Welcome to Scholar Publishing Group

International Journal of Big Data Intelligent Technology, 2026, 7(2); doi: 10.38007/IJBDIT.2026.070210.

Research on Real-time Data Synchronization Technology for Database Migration Based on CDC

Author(s)

Dong Shao

Corresponding Author:
Dong Shao
Affiliation(s)

Pepperdine Graziadio Business School, Pepperdine University, Malibu, 90263, California, USA

Abstract

To address the challenge of balancing incremental backlog, transaction freshness, and commit overhead in online database migration, an age-constrained online micro-batch synchronization method, AOMB, is proposed. This method uses persistent complete CDC transactions as scheduling units, leverages recursive least squares to estimate the target server's service capacity, and combines a physical backlog queue, a virtual transaction age queue, and a Binlog switching barrier to select batches. Controlled-repetition experiments were conducted on a MySQL-Debezium-PostgreSQL link, and comparisons were made with fixed-batch, queued AIMD, MPA-TB, and offline Oracle methods. Results show that AOMB reduces switching time by 8.1%–10.9% and transaction age violation rate by 35.5%–39.6% under three types of unknown loads. However, its advantages diminish with increasing commit barrier wait times under high hotspot and capacity boundary conditions. The study demonstrates that this method can improve the switching timeliness of CDC migration, but its effectiveness is constrained by service margin and conflict levels.

Keywords

Change data capture; Database migration; Real-time data synchronization; Online micro-batch processing; Transaction age; Queue optimization

Cite This Paper

Dong Shao. Research on Real-time Data Synchronization Technology for Database Migration Based on CDC. International Journal of Big Data Intelligent Technology (2026), Vol. 7, Issue 2: 94-104. https://doi.org/10.38007/IJBDIT.2026.070210.

References

[1] Fragkoulis M, Carbone P, Kalavri V, Katsifodimos A. A survey on the evolution of stream processing systems[J]. The VLDB Journal, 2024, 33(2): 507-541. DOI: 10.1007/s00778-023-00819-8.

[2] Zhang S, Soto J, Markl V. A survey on transactional stream processing[J]. The VLDB Journal, 2024, 33(2): 451-479. DOI: 10.1007/s00778-023-00814-z.

[3] Zhao J, Mao Y, Yang Z, Liu H, Zhang S. Scalable transactional stream processing on multicore processors[J]. IEEE Transactions on Knowledge and Data Engineering, 2025, 37(7): 4254-4269. DOI: 10.1109/TKDE.2025.3556741.

[4] CAO H, HUANG C, LIU S, et al. Aion: Live migration for in-memory databases with zero downtime and reduced redundant data transfer[J]. Data Science and Engineering, 2025, 10(2): 212-229. DOI: 10.1007/s41019-024-00276-5.

[5] RUCCO C, LONGO A, SAAD M. Enhancing data ingestion efficiency in cloud-based systems: A design pattern approach[J]. Data Science and Engineering, 2026, 11(2): 287-302. DOI: 10.1007/s41019-025-00300-2.

[6] Wang, B. (2025). Methods of Load Optimization for Computer Systems Based on Physical Principles.

[7] Guo, X. (2025, March). Research on Blockchain-Based Financial AI Algorithm Integration Methods and Systems. In 2025 IEEE International Conference on Electronics, Energy Systems and Power Engineering (EESPE) (pp. 816-821). IEEE.

[8] Guo, X. (2025, April). Research on Financial Trading Algorithms Based on Deep Reinforcement Learning. In 2025 IEEE 3rd International Conference on Control, Electronics and Computer Technology (ICCECT) (pp. 1711-1715). IEEE.

[9] Yu, X. (2026). Exploration of Multi-Channel Conversion Path Optimization Methods Based on A/B Testing.

[10] Jing, X. (2025, November). In-Depth Analysis of the Optimization Decision Model of Financial Investment Portfolio Based on Machine Learning Algorithms in Python Scikit-Learn Library. In International Conference on Computational Technologies for Research in Data Analytics (pp. 130-140). Cham: Springer Nature Switzerland.

[11] Liu, X. (2026). Research on the Application of Artificial Intelligence in Consumer Privacy Data Protection. Procedia Computer Science, 279, 956-965.

[12] Liu, X. (2026). Construction of Consumer Data Privacy Protection System Based On Blockchain Technology. Procedia Computer Science, 282, 2082-2091.

[13] Han, W. (2025, November). An Empirical Study on Building a High-precision Forecasting System for Financial Market Trends Based on the PyTorch Deep Learning Framework. In International Conference on Computational Technologies for Research in Data Analytics (pp. 518-528). Cham: Springer Nature Switzerland.

[14] Yuan, Y. (2026). Research on Memory Management and Dynamic Reasoning Methods for Intelligent Agents Based on Large Language Models.

[15] Yang, Y. (2026). Causal Inference-based Identification of Incremental Effects in Digital Advertising and Optimization Pathways for Resource Allocation. Advances in Computer and Communication, 7(2).

[16] Sun, L. (2026). Consistency Optimization Strategies for Distributed Advertising Systems Under Cross-regional Deployment. Advances in Computer and Communication, 7(2).