Article information

2016 , Volume 21, ¹ 3, p.70-79

Smagin S.I., Loupian E.A., Sorokin A.A., Burtsev M.A., Korolev S.P., Proshin A.A., Kramareva L.S.

Development of methods and technologies for distributed storage and processing of the remote sensing data for environment monitoring purposes

Purpose. In recent years an increased number of tools with improved qualitative characteristics for remote environmental observation along with developed of satellite data analysis techniques and expanded fields of their application have resulted in a sharp growth of the number and volumes of storage of satellite data archives. At the same time, the creation of technologies and software packages for storage of archives of such large data and their distributed processing and use are somewhat lagged behind. This reduces the ability and efficiency for application of remote sensing data and their processing results for the solution of research problems on a large scale. The authors of the paper conduct studies and develop contemporary techniques and algorithms for storage, processing and analysis of super-large volumes of remote sensing data using the capabilities of cloud computing environments.

Methodology. This work implies the creation of a programming platform which provides tools that allow searching for (transparent to user or external information system) and receiving of the necessary instrumental data and their processing results from hundreds of thematic servers and repositories distributed over the territory of Russia. The implementation of this work is associated with an implementation of technologies in compliance with the following directions:

∙ distributed archive storage of Earth remote sensing (ERS) data and their processing results;

∙ providing fail-safe distributed archive data storage;

∙ data presentation and analysis based on RESTful services.

Originality. Development of the UNISAT system architecture for distributed archive storage of Earth remote sensing (ERS) data and their processing results can be considered as the first result of our work to implement this Project. Besides, it realizes technologies on metadata exchange between the distributed nodes, and also, a single universal directory for data type storage as well as techniques of construction of different “virtual” information products on their basis.

Findings. The paper describes the features related to the solution of above listed problems in Russia, estimation of possible technical tools which allow us to increase the efficiency and speed of operations with the data, and also, the first results of our work in this direction.

[full text]
Keywords: cloud technologies, science data sets, database, remote sensing, information system, distributed data storage and processing, RESTful web-service, environment

Author(s):
Smagin Sergey Ivanovich
Dr. , Correspondent member of RAS, Professor
Position: Director
Office: Computer Center FEB RAS
Address: 680000, Russia, Khabarovsk
Phone Office: (4212) 22 72 67
E-mail: smagin@ccfebras.ru
SPIN-code: 2419-4990

Loupian Evgeniy Arkad'evich
Dr.
Position: Deputy general director
Office: Space Research Institute of the Russian Academy of Sciences
Address: 117997, Russia, Moscow, Profsoyuznaya Street, 84/32
Phone Office: (495) 333-53-13
E-mail: evgeny@d902.iki.rssi.ru
SPIN-code: 5841-7796

Sorokin Aleksei Anatolyevich
PhD.
Position: Leading research officer
Office: CC FEB RAS
Address: 680000, Russia, Khabarovsk, 65, Kim Yu Chen str.
Phone Office: (4212) 703913
E-mail: alsor@febras.net
SPIN-code: 1767-2259

Burtsev Mikhail Aleksandrovich
PhD.
Position: Research Scientist
Office: Space Research Institute RAS
Address: 117997, Russia, Moscow, 65, Kim Yu Chen str.
Phone Office: (495) 333-52-12
E-mail: burcev@d902.iki.rssi.ru
SPIN-code: 7049-6871

Korolev Sergey Pavlovich
Position: Research Scientist
Office: CC FEB RAS
Address: 680000, Russia, Khabarovsk, 65, Kim Yu Chen str.
Phone Office: (4212) 703913
E-mail: serejk@febras.net
SPIN-code: 5884-4506

Proshin Andrey Alekseevich
PhD.
Position: Senior Research Scientist
Office: Space Research Institute RAS
Address: 117997, Russia, Moscow, 65, Kim Yu Chen str.
Phone Office: (495) 333-52-12
E-mail: proshin_a@mail.ru
SPIN-code: 6474-5536

Kramareva Lyubow Sergeevna
Position: Director
Office: Far-Eastern Center of State Research Center for Space Hydrometeorology Planeta
Address: 680000, Russia, Khabarovsk, 18, Lenina st.
Phone Office: (4212) 21-42-21
E-mail: kramareva@dvrcpod.ru
SPIN-code: 6216-0029

References:
[1] Loupian, E., Savin, I., Bartalev, S., Tolpin, V., Balashov, I., Plotnikov D. Satellite service for vegetation monitoring VEGA. Current Problems in Remote Sensing of the Earth from Space. 2011; 8(1):190–198. (In Russ.)

[2] Smagin, S.I., Loupian, E.A., Sorokin, A.A., Bourtsev, M.A., Verkhoturov, A.L., Girina, O.A., Efremov, V.Yu., Kramareva, L.S., Proshin, À.À., Tolpin, V.A. Information system for scientific research in various fields of expertise using satellite data in the region of the Far East of Russia. Current Problems in Remote Sensing of the Earth from Space. 2013; 10(1):277–291. (In Russ.)

[3] Efremov, V.Y., Girina, O.A., Kramareva, L.S., Loupian, E.A., Manevich, A.G., Melnikov, D.V., Matveev, A.M., Proshin, A.A., Sorokin, A.A., Flitman, E.V. Creating an Information Service “Remote Monitoring of Active Volcanoes of Kamchatka and the Kuril Islands”. Current Problems in Remote Sensing of the Earth from Space. 2012; 9(5):155–170. (In Russ.)

[4] Lupyan, E.A., Milekhin, O.E., Antonov, V.N., Kramareva, L.S., Burtsev, M.A., Balashov, I.V., Tolpin, V.A., Solov’ev, V.I. Problems of data organization and use within the Planeta Research Center for space hydrometeorology United Satellite Data Access System. Meteorologiya i Hydrologiay. 2014; (12):89–97. (In Russ.)

[5] Loupian, E.A., Matveev, A.A., Uvarov, I.A., Bocharova, T.Yu., Lavrova, O.Yu., Mityagina, M.I. The satellite service See the Sea — a tool for the study of oceanic phenomena and processes. Current Problems in Remote Sensing of the Earth from Space. 2012; 9(2):251–262. (In Russ.)

[6] Samet, R., Tural, S. Web based real-time meteorological data analysis and mapping information system. International Journal of Education and Information Technologies. 2010; 4(4):187–196.

[7] Shokin, Yu.I., Zhizhimov, O.L., Pestunov, I.A., Sinyavskiy, Yu.N., Smirnov, V.V. Distributed information-analytical system for searching, processing and analysis of spatial data. Computational Technologies. 2007; 12(spec. issue. 3):108–115. (In Russ.)

[8] Antonopoulus, N., Gillam, L. Cloud computing: Principles, systems and applications. Springer; 2010: 396.

[9] Pallickara, S., Pierce, M. Scientific data management in the cloud: A survey of technologies, approaches and Challenges. Eds. Furht, B., Escalante, A. Handbook of Cloud Computing. Springer Science+Business Media; 2010: 517–533.

[10] Zhizhin, Ì., Medvedev, D., Mishin, D., Poyda, A., Novikov, A. Transparent data cube for spatiotemporal data mining and visualization. Eds. Fiore, S., Aloisio, G. Grid and Cloud Database Management. Springer; 2011: 307–330.

[11] Ramapriyan, H.K. Development, operation and evolution of EOSDIS — NASA’s major capability for managing Earth science data. CENDI/NFAIS Workshop on Repositories in Science & Technology: Preserving Access to the Record of Science. Washington; November 30, 2011. http://www.cendi.gov/activities/11_30_2011_CENDI_NFAIS_FLICC.html

[12] Loupian, E.A., Savorsky, V.P., Shokin, Yu.I., Aleksanin, A.I., Nazirov, R.R., Nedolugko, I.V., Panova, O.Yu. Up-to-date approaches and technology arrangement of Earth observation data applications aimed to solve scientific tasks. Current Problems in Remote Sensing of the Earth from Space. 2012; 9(5):21–44. (In Russ.)

[13] Miklashevich, S.E., Balashov, I.V., Burcev, M.A., Efremov, V.Yu., Mazurov, A.A., Matveev, A.M., Proshin, A.A., Radchenko, M.V., Flitman, E.V. Complex system for the receiving, processing, archiving and distribution of satellite data and products of thematic processing. Current Problems in Remote Sensing of the Earth from Space. 2012; 9(4):47–56. (In Russ.)

[14] Savorskiy, V.P., Lupyan, E.A., Kaevitser, V.I., Smirnov, M.T., Ermakov, D.M., Balashov, I.V., Bourtsev, M.A., Efremov, V.Yu., Proshin, A.A., Tolpin, V.A., Uvarov, I.A., Flitman, E.V. Distributed infrastructure enabling effective integration of earth observation information resources for collective solution of archiving, searching, processing and eo data analyzing tasks. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences. XXII ISPRS Congress, 25 August – 01 September 2012, Melbourne, Australia. 2012; (Volume XXXIX-B4):237-242.

[15] Loupian, E.A., Mazurov, A.À., Nazirov, R.R., Proshin, A.A., Flitman, E.V., Krasheninnikova, Y.S. Technologies for building remote monitoring information systems. Current Problems in Remote Sensing of the Earth from Space. 2011; 8(1):26–43. (In Russ.)

[16] Antonov, A.V., Bourtsev, M.A., Efremov, V.Yu., Kalashnikov, A.V., Kramareva, L.S., Krasheninnikova, Yu.S., Loupian, E.A., Matveev, A.M., Proshin, A.A., Flitman, E.V. Design of the union catalogue of distributed satellite data archives of various centers. Current Problems in Remote Sensing of the Earth from Space. 2010. 7(2):84–89. (In Russ.)

[17] Bourtsev, M.A., Loupian, E.A., Mazurov, A.A., Nazipov, R.R., Proshin, A.A., Flitman, E.V. Technology of automated satellite data archiving and representing systems development. Proc. of the 7th Intern. Symp. on Reducing Costs of Spacecreft Ground Systems and Operations (RCSGSO). 11–15 June 2007. Moscow. Russian Federation (ESA SP-648, July 2007); 2007: 3b2-1–3b2-4.

[18] Efremov, V.Yu., Loupian, E.A., Mazurov, A.À., Proshin, A.A., Flitman, E.V. Technology of automated satellite data storage systems. Current Problems in Remote Sensing of the Earth from Space. 2004; 1(1):437–443. (In Russ.)

[19] Sorokin, A.A., Korolev, S.P., Smagin, S.I., Polyakov, A.N. Layout of a resilient informational system for cloud storage of science data sets. Computational Technologies. 2013; 18(1):87–95. (In Russ.)

[20] Khanchuk, A.I., Sorokin, A.A., Smagin, S.I., Korolev, S.P., Makogonov, S.V., Tarasov, A.G., Shestakov, N.V. Development of information and telecommunication systems in the Far Eastern Branch of RAS. Information Technologies & Computer Systems. 2013; (4):45–57. (In Russ.)

[21] Sorokin, A.A., Korolev, S.P., Makogonov, S.V., Shestakov, N.V. The use of cloud computing for storing and processing instrumental data from seismological observations networks. Proceedings of the International Conference on Computer Science and Environmental Engineering (CSEE 2015), Beijing, May 17–18. 2015: 546–552.

[22] Sorokin, A.A., Korolev, S.P., Urmanov, I.P., Verkhoturov, A.L., Shestakov, N.V., Girina, O.A. Information system for work with instrumental observations data for research and monitoring of natural hazards in the Far East Russia. Proceedings of the Russ. Conference “Geodynamic Processes and Natural Disasters. Experience Neftegorsk”. Vladivostok: Dal‘nauka; 2015; (2): 443–447. (In Russ.)

[23] Korolev, S.P., Sorokin, A.A., Verkhoturov, A.L., Konovalov, A.V., Shestakov, N.V. Automated information system for instrument data processing of the regional seismic observation network of FEB RAS. Seismic Instruments. 2014; 50(3):30–41. (In Russ.)


Bibliography link:
Smagin S.I., Loupian E.A., Sorokin A.A., Burtsev M.A., Korolev S.P., Proshin A.A., Kramareva L.S. Development of methods and technologies for distributed storage and processing of the remote sensing data for environment monitoring purposes // Computational technologies. 2016. V. 21. ¹ 3. P. 70-79
Home| Scope| Editorial Board| Content| Search| Subscription| Rules| Contacts
ISSN 1560-7534
© 2024 FRC ICT