|
Article information
2026 , Volume 31, ¹ 4, p.93-104
Nikolaev E.P., Voskoboinikov M.L.
Agent-based automating the building, configuration and deployment of scientific applications in a heterogeneous distributed computing environment
The DevOps supports for workflow-based scientific applications in a heterogeneous distributed computing environment is a challenge. It is characterized by significant overhead costs for manual software deployment and the lack of mechanisms for continuous runtime environment monitoring and support for fault tolerance of computing processes. To automate and decompose the DevOps pipeline, we propose specialized execution agents. Each agent is responsible for a specific stage such as node preparation, containerd installation, Kubernetes cluster initialization, or HTCondor deployment. After the deployment is complete, monitoring agents deployed on critical nodes are activated. These agents perform regular cluster health checks using native utilities, collect metrics, and initiate recovery actions to restore cluster functionality when failures are detected. Comparative analysis of overheads in preparing and conducting experiments demonstrates the substantial overhead reduction in preparing and conducting experiments due to the DevOps automation in comparison with a manual management. The most reduction is achieved during the HTCondor file configuration and the implementation of reliable network interactions between cluster nodes. The proposed approach allows us to automate the initial deployment and ensures continuous manageability, ability to observe, and fault tolerance throughout the entire lifecycle of computational experiments. The DevOps pipeline decomposition into isolated, observable, and controllable stages enhances the reproducibility, reliability, and portability of scientific applications across diverse configurations.
Keywords: scientific application, software building, configuration, and deployment, orchestration, containerization, agent
Author(s): Nikolaev Evgeny Petrovich Position: researcher Office: Institute for System Dynamics and Control Theory Siberian Branch of RAS Address: 664033, Russia, Irkutsk, Lermontov St. 134
Phone Office: (3952) 45-30-17 E-mail: zhenya_nikolaev_1995@list.ru SPIN-code: 8239-1735Voskoboinikov Mikhail Leontevich Position: Junior Research Scientist Office: Institute for System Dynamics and Control Theory Siberian Branch of RAS Address: 664033, Russia, Irkutsk, Lermontov St. 134
Phone Office: (3952) 45-30-17 E-mail: mikev1988@mail.ru SPIN-code: 3417-0258 Bibliography link: Nikolaev E.P., Voskoboinikov M.L. Agent-based automating the building, configuration and deployment of scientific applications in a heterogeneous distributed computing environment // Computational technologies. 2026. V. 31. ¹ 4. P. 93-104
|