A Microservice Architecture for the Processing of Large Geospatial Data in the Cloud

Abstract

With the growing number of devices that can collect spatiotemporal information, as well as the improving quality of sensors, the geospatial data volume increases con- stantly. Before the raw collected data can be used, it has to be processed. Current- ly, expert users are still relying on desktop-based Geographic Information Systems to perform processing workflows. However, the volume of geospatial data and the complexity of processing algorithms exceeds the capacities of their workstations. There is a paradigm shift from desktop solutions towards the Cloud, which offers virtually unlimited storage space and computational power, but developers of pro- cessing algorithms often have no background in computer science and hence no expertise in Cloud Computing. Our research hypothesis is that a microservice architecture and Domain-Specific Languages can be used to orchestrate existing geospatial processing algorithms, and to compose and execute geospatial workflows in a Cloud environment for efficient application development and enhanced stakeholder experience. We present a soft- ware architecture that contains extension points for processing algorithms (or mi- croservices), a workflow management component for distributed service orchestra- tion, and a workflow editor based on a Domain-Specific Language. The main aim is to provide both users and developers with the means to leverage the possibilities of the Cloud, without requiring them to have a deep knowledge of distributed com- puting. In order to conduct our research, we follow the Design Science Research Methodology. We perform an analysis of the problem domain and collect require- ments as well as quality attributes for our architecture. To meet our research objec- tives, we design the architecture and develop approaches to workflow management and workflow modelling. We demonstrate the utility of our solution by applying it to two real-world use cases and evaluate the quality of our architecture based on defined scenarios. Finally, we critically discuss our results. Our contributions to the scientific community can be classified into three pillars. We present a scalable and modifiable microservice architecture for geospatial pro- cessing that supports distributed development and has a high availability. Further, we present novel approaches to service integration and orchestration in the Cloud as well as rule-based and dynamic workflow management without a priori design-time knowledge. For the workflow modelling we create a Domain-Specific Language that is based on a novel language design method. Our evaluation results support our hypothesis. The microservice architectural style enables efficient development of a distributed system. The Domain-Specific Language and our approach to service integration enhance stakeholder experience. Our work is a major step within the paradigm shift towards the Cloud and opens up possibilities for future research.