Documentation of IQmulus products for developers: infrastructural background, development requirements, software components, etc.
Organizations involved: Fraunhofer (Germany), MOSS (Germany), FÖMI (Hungary)
Figure. IQMULUS ARCHITECTURE
The IQmulus infrastructure uses Cloud technology to parallelize geospatial processing workflows. The system consists of multiple components. The user interface (GUI) contains a data browser, a workflow editor and a visualization component. In addition to that, there are a number of processing services running in the Cloud and a component called Job Manager which runs the processing workflows and oversees their execution.
As wide range of possible users with diverse backgrounds are the audience for the IQmulus infrastructure, special emphasis has been given to minimal complexity and maximal navigability.
Data Browser. The Data Browser provides basic functionality for users to browse existing data and upload new data into the IQmulus cloud. Data must always come in pairs of files and metadata information. The upload of the two are only loosely coupled in the prototype as will be described in the paragraphs below.
Workflow Editor. The workflow editor can be used to select existing workflows or to create a new one using a domain-specific language (DSL).
Wizard. The Wizard leads the user through the process of running a workflow in the infrastructure. It follows a basic three step procedure:
1. Create a workflow or select an existing one
2. Select the data to be processed
3. Insert the data into the workflow and launch it
The wizard was implemented to be a convenient tool for a wide range of users and is, as all other IQmulus components are, fully responsive and as such usable on mobile devices (tablets, phones) as well as laptops and desktop computers. The information content of each page was designed with that in mind. See the Figure below.
Job Manager and Processing services. The Job Manager is one of the central components of the IQmulus system. The Wizard sends workflows selected by the user to the Job Manager. It then interprets these workflows and calculates the best strategy to execute it in the Cloud. Finally, it runs the processing services on the individual compute nodes in the Cloud and oversees their execution. The processing services read input files from a distributed file system (DFS) and write their results back to the DFS.
Results. The results page that offers the users a modern and convenient dashboard to manage processing results, see the second below. It offers an overview as well as detailed job summaries, data download and visualization options. By default the dashboard will load and display all jobs. Status filters allow the user to limit the results to jobs of a particular status (e.g. success, failed). A free text search allows the user to find workflows by keyword. Details of each job are displayed in the respective box on the right.
Visualisation. The user interface also includes a visualization component. It can be used to display input data stored on the Cloud or to display the results of a processing workflow. We provide two different visualization clients. A web-based client that runs in the Internet browser and a high-performance desktop client that needs to be downloaded and installed before it can be used. The users can select the right client based on the workflow they executed and the results they want to display.
DATA SELECTOR - SEARCHING DATA IN THE IQMULUS DATA CLOUD.
FIGURE. RESULTS DASHBOARD