Container Fault Detection System CPDS (Container Problem Detect System) is a container cluster fault detection system designed and developed by Beijing Yinsi Software Co., Ltd. (hereinafter referred to as "Yinsi Software"). This software system realizes monitoring and identification of container TOP faults and sub-health status.
In November 2022, Meditation Software officially opened the project to the openEuler community.
+ + + + + + + + + + +
Background introduction:
With the wave of digital development of enterprises in recent years and the popularization of cloud-native technologies, low latency and high concurrency online scenarios frequently appear in the daily operations of enterprises, and the demand for business innovation is also forcing enterprises to continuously use emerging technology means. Nowadays, the container technology is widely used in artificial intelligence , big data, edge computing and other scenarios. As a lightweight computing carrier, it gives high flexibility and agility to more scenarios.
business scale has grown, the scale of container clusters has continued to expand, and the pressure on IT operation and maintenance has also increased proportionally. Business interruptions caused by various software and hardware failures have become one of the important factors affecting stability. At present, the industry's detection solutions for container cluster failures are mainly based on cluster component status detection, service entrance monitoring, custom interface detection and activation, etc., which have certain limitations and it is difficult to detect and identify the sub-health status of the service. The handling methods also lack the formulation of fault diagnosis and implementation strategies, and it is difficult to deal with some key and core faults.
In response to the current problems in the cloud native field, Meditation Software (LINX SOFTWARE) launched the CPDS (Container Fault Detection System) open source project. The system developed by this project realizes fault detection, diagnosis and policy execution of container clusters, solving a core pain point in the industry.
+ + + + + + + + + + +
Technical route:
Container cluster exceptions are generally three categories, namely: cluster basic service exceptions, cluster OS exceptions, and business service exceptions.
cluster basic service exception class problem, using systemd, initv and other technologies to start, monitor and execute related key services.
cluster OS exception class problem, implement node agents on the host, collect, report and implement policies related information such as node network, system calls, disk LVM, etc.
business service exception type problem, a cross-NS proxy is set up in nodes and containers in a non-invasive way, and comprehensive monitoring, analysis and exception detection are carried out for the application status, resource consumption, key system function execution status, and io execution status in the container. For example, in which CPU is occupancy, multiple parameters and indicators (irq, iowait, etc.) are analyzed in detail to locate the direction of the problem. At the same time, the data collected from system calls, networks, disks, etc. are combined to analyze them to locate the cause of the failure and take corresponding measures. At the same time, multi-dimensional analysis of execution results, time-consuming and other multi-dimensional analysis is carried out for related system calls applied in the business container, and potential problems such as container sub-health, and corresponding measures are taken to ensure the stable operation of the business container.
+ + + + + + + + + +
System architecture:
CPDS system technical architecture is shown in the figure below. The information collection module collects key data from multiple levels such as cluster basic services, cluster systems, business container services, etc. through nodes and container information collection programs, and reports the abnormality detection module; after completing data preprocessing, the abnormality detection module performs abnormality detection based on abnormal rules, and after completion, the detection result data is uploaded to the diagnostic module; the diagnostic module performs fault/sub-health diagnosis of nodes and business containers based on diagnostic rules, and displays visual diagnostic results through the user interaction module.
+ + + + + + + + + + +
Future planning:
CPDS is developed and maintained by CloudNative SIG, and the first R version is planned to be released in the second half of 2023 to provide support for openEuler-22.03-LTS.
Project address:
https://gitee.com/openeuler/Cpds
+ + + + + + + + +
About Yinsi
Beijing Yinsi Software Co., Ltd. (hereinafter referred to as "Yinsi Software") was founded by "Chinese Linux pioneer" Dr. Gong Min. It is an advanced domestic security operating system manufacturer. With the mission of "realizing the domestic and secure basic software of the country's large-scale infrastructure industry", it is committed to the research and development and production of "high security, high performance, and high stability" basic operating system software and related products and services, ensuring the safe and stable operation of production and key business systems at the operating system level. It has now become the backbone of domestic large-scale basic industries domestic operating systems.
meditation software has a series of products such as secure operating system, container system, distributed storage, virtualization management platform, secure cloud desktop system, big data platform, trusted verification system, intranet host security supervision software, network high-availability software, Windows application operation platform, etc.
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-