As part of the scientific work, an analysis of the problems of a previously developed prototype of a water cannon robot operating on the basis of a multithreaded asynchronous video analysis kernel and corresponding algorithms for filtering incoming frames to identify areas of the environments, with the ability to process neural network models of color segmentation of the flame in real time.
analysis, android smartphone, water cannon
Special attention is currently being paid to the problem of preventing natural disasters. Forest fires are the most common catastrophe causing serious damage to the Russian economy. It should be noted that the most often guilty in the occurrence of these phenomena is a person and careless handling of fire. Also, one cannot ignore the fact that our country has vast forests, and there is not enough professional personnel to monitor them. Therefore, the scientific community is faced with the task of developing automated means of protecting and counteracting forest fires. One of the promising areas of computational cybernetics is the development of new methods of detection and analysis behind flames. In case of forest fires, the supply of water to the hearth of the flame without taking into account its structure is ineffective, therefore it is necessary to develop an algorithm for extracting the zones of fire vulnerability on video.
As part of the scientific work, a prototype of a robot-water cannon was developed [1], shown in fig.1.
Fig. 1 – Robot water cannon prototype
During testing of the robot shown in fig. 2, the problem of overexposure of the flame in the frame area was identified.
Fig. 2 – Water cannon robot testing
Real-time fire analysis [2] and correct display of the color spectrum of the flame on video are key factors in the effectiveness of fire extinguishing. In the implementation of video analytics as part of the design of the robot, an algorithm was developed to search for flame zones without specifying its color configuration [3]. To enable it to work in real time on the Raspberry PI 3 Model B microprocessor architecture, the core of asynchronous multithreaded video analysis was developed [4]. In order to qualitatively improve the results of flame detection, approaches to flame segmentation were applied using modern convolutional neural networks of the UNet class [5], and its modification was developed for the problem of segmentation of objects of the same kind on the basis of wUUNet [6]. Objects of the same kind within the framework of our task are the contours of the flame, and the sign that distinguishes the types of flame from each other is the color (Yellow, orange and red). This division was made in order to follow the rule of extinguishing a fire in the lower zone of the flame with a temperature of 600K [7], which corresponds to the red color in the video. In order for this neural network algorithm to work effectively, it is necessary to select a device capable of high-quality photographing of the flame and process the frame in real time. To do this, we took the initial configuration in the form of a Raspberry PI 3 Model B board and a Pi camera module, and opposed it with the Android smartphone Realme X2 Pro, which has a 64 MPix camera and a system based on a Qualcomm Snapdragon 855 chip. The comparison results between the frames of the flame fixing devices are shown in fig. 3-5.
Fig. 3 – Flame video recording on Raspberry PI 3 Camera module with auto setting
The Raspberry PI Camera module supports automatic shooting mode, in which the flame contour appears to be overexposed as a circle, which does not correspond to the successful detection of vulnerable zones. Experimentally, the "spotlight" operating mode was found, in which a bright object becomes clearly visible. However, in this case, we lose information about the environment surrounding the flame, which is important for the operation of both the convolutional neural network and the subsequent stages of video processing in order to search for additional information.
Fig. 3 – Flame video recording on Raspberry PI 3 Camera module with spotlight setting
Fig. 5 – Flame video shooting on Realme x2 Pro
Video shooting on the Realme x2 pro smartphone is optimal both in terms of obtaining correct information about the flame and its surroundings.
The second stage of comparison between the considered software and hardware configurations is to obtain performance characteristics of the computations of the neural network model UNet above the frame, in order to find clear contours of a flame of different colors. Neural networks are massively parallel algorithms for matrix calculations, so the presence of a graphics core in a mobile phone chipset is an important advantage over the lack of one on a Raspberry PI 3 Model B microcomputer. Indeed, Table 1 shows a significant advantage of using a mobile phone.
|
Device |
Processing time for 1 frame, UNet model |
FPS indicator |
|
Raspberry PI 3 Model B |
0.87 sec. |
1.14 |
|
Realme x2 Pro |
0.25 sec. |
40 |
Tab. 1 – Device performance characteristics
Thus, it was found that the use of a modern mobile phone shows better results in comparison with the Raspberry PI3 Model B microcomputer in the task of filming and timely segmentation of the flame by color.
1. Bochkov V.S., Kataeva L.Yu., Belotserkovskaya I.E., Ilicheva M.N. Prototype of forest fire suppression robotics system based on exclusion of flame configuration from monocular video sequence. ARPN Journal of Engineering and Applied Sciences 2019, 14 (9). 1719-1730.
2. Kataeva L.Y., Maslennikov D.A., Loshchilova. N.A. On the laws of combustion wave suppression by free water in a homogeneous porous layer of organic combustive materials. Fluid dynamics 2016, 51 (3), 389-399.
3. Bochkov V.S., Kataeva L.Yu., Maslennikov D.A. Algorithm for searching for vulnerable areas of fire using video stream analysis. XXIX International Scientific and Practical Conference dedicated to the 80th anniversary of the FSBI VNIIPO EMERCOM of Russia Conference materials. In 2 parts. 2017. P. 395-400.
4. Bochkov V.S., Kataeva L.Yu., Maslennikov D.A. Implementation of an algorithm for searching for vulnerable areas of fire in the software architecture of a multithreaded asynchronous video analytics core. XXIX International scientific-practical conference dedicated to the 80th anniversary of FSBI VNIIPO EMERCOM of Russia Materials of the conference. In 2 parts.2017. P. 385-389.
5. Ronneberger O., Fisher P., Brox T. U-Net: Convolutional networks for biomedical image segmentation. arXiv:1505.04597 [cs.CV] 2015.
6. Bochkov V.S., Kataeva L.Y. WUUNet: Advanced fully convolutional neural network for multiclass fire segmentation. Symmetry, 2021, 13(1), P. 1-18, 98
7. Bochkov V.S., Belyaev I.V., Maslennikov D.A., Kataeva L.Yu., Iliicheva M.N. Analytical solution of the problem of combustion wave propagation in a homogeneous porous layer of organic combustible materials
