OR/14/022 Introduction - Revision history

Dbk: /* Coupler use cases and requirements gathered from BGS staff */

2015-08-06T13:36:26Z

Coupler use cases and requirements gathered from BGS staff

← Older revision		Revision as of 14:36, 6 August 2015
Line 16:		Line 16:

	==Coupler use cases and requirements gathered from BGS staff==		==Coupler use cases and requirements gathered from BGS staff==
	In 2010 the BGS produced the Dream Scoping study report (Giles et al., 2010<ref name="Giles">~~Giles~~, ~~Jeremy~~ R A, et al. 'Data, and research for applications and models (DREAM): scoping study report.' (2010). </ref>), as part of the research for this report a wide range of BGS scientists, responsible for answering questions raised by clients, were asked what they required from a model linkage solution, below are a selection of those responses:		In 2010 the BGS produced the Dream Scoping study report (Giles et al., 2010<ref name="Giles">GILES, J R A, et al. 'Data, and research for applications and models (DREAM): scoping study report.' (2010). </ref>), as part of the research for this report a wide range of BGS scientists, responsible for answering questions raised by clients, were asked what they required from a model linkage solution, below are a selection of those responses:

	: ''“As a geologist focussed on the urban environment I want an environmental modelling platform to act as an effective communication tool, perhaps through visual representations of processes, so that others, including non-geologists, can better understand the model.”''		: ''“As a geologist focussed on the urban environment I want an environmental modelling platform to act as an effective communication tool, perhaps through visual representations of processes, so that others, including non-geologists, can better understand the model.”''

Dbk: /* Structure of these articles */

2015-08-06T13:34:55Z

Structure of these articles

← Older revision		Revision as of 14:34, 6 August 2015
Line 40:		Line 40:

	==Structure of these articles ==		==Structure of these articles ==
	The following articles describe in detail the [OR/14/022 Description of dynamic (run-time) approaches \| dynamic (run-time) approaches for atmospheric and hydrological approaches], which is followed by a summary of [OR/14/022 Data standards for one way, static transfer of data \|		The following articles describe in detail the [[OR/14/022 Description of dynamic (run-time) approaches \| dynamic (run-time) approaches for atmospheric and hydrological approaches]], which is followed by a summary of [[OR/14/022 Data standards for one way, static transfer of data \|
	data standards for one-way, static transfer of data]. Article [OR/14/022 Comparison of approaches]compares the different approaches and the findings of are summarised in [OR/14/022 Summary and recommendations] along with providing recommendations for the next stage of work.		data standards for one-way, static transfer of data]]. Article [[OR/14/022 Comparison of approaches]] compares the different approaches and the findings of are summarised in [[OR/14/022 Summary and recommendations]] along with providing recommendations for the next stage of work.

			==References==


	[[category:OR/14/022 Couplers for linking environmental models: Scoping study and potential next steps \| 02]]		[[category:OR/14/022 Couplers for linking environmental models: Scoping study and potential next steps \| 02]]

Dbk at 13:33, 6 August 2015

2015-08-06T13:33:47Z

Dbk: /* The need for couplers */

2015-05-22T09:53:21Z

The need for couplers

← Older revision		Revision as of 10:53, 22 May 2015
Line 11:		Line 11:
	The technologies used for coupling models vary in the level of “intrusiveness”, which can be defined as the amount of work required to make a component “couplable” (Lawrence et al., Manuscript). The coupling technologies can be divided into: monolithic, component-based, communication-based, and scheduled (Dunlap et al., 2013). The monolithic approach requires combining code from multiple models into one code (Dunlap et al., 2013). The component-based approach introduces the concept of standard interfaces. In this approach each model, called a “component”, has: an interface to communicate with other models, a structure in compliance with predefined criteria, and performs a distinct function (Dunlap et al., 2013, Lu 2011). In communication and scheduled approaches models are independent (Dunlap et al., 2013, Lu 2011). The communication approach requires embedding library calls within the model's code for sending and receiving data (Dunlap et al., 2013). In the scheduled approach the output from one model is used as an input to the next one, thus the models do not affect each other during the execution (Dunlap et al., 2013).		The technologies used for coupling models vary in the level of “intrusiveness”, which can be defined as the amount of work required to make a component “couplable” (Lawrence et al., Manuscript). The coupling technologies can be divided into: monolithic, component-based, communication-based, and scheduled (Dunlap et al., 2013). The monolithic approach requires combining code from multiple models into one code (Dunlap et al., 2013). The component-based approach introduces the concept of standard interfaces. In this approach each model, called a “component”, has: an interface to communicate with other models, a structure in compliance with predefined criteria, and performs a distinct function (Dunlap et al., 2013, Lu 2011). In communication and scheduled approaches models are independent (Dunlap et al., 2013, Lu 2011). The communication approach requires embedding library calls within the model's code for sending and receiving data (Dunlap et al., 2013). In the scheduled approach the output from one model is used as an input to the next one, thus the models do not affect each other during the execution (Dunlap et al., 2013).

			The coupling technologies can be formally divided into: coupling libraries, coupling frameworks, and workflows (Lawrence et al., Manuscript, Dunlap et al., 2013). Libraries provide concrete solution fragments (Lawrence et al., Manuscript); they minimise the amount of code changes required to make a model couplable, typically allowing it to act as independent executable and merely to exchange data at appropriate locations and times (Dunlap et al., 2013). Frameworks use standard interfaces for communication with the components, which must comply with the interfaces' calling conventions (Dunlap et al., 2013). Consequently that components must be structured in accordance with a predefined architectural design (Dunlap et al., 2013). Workflow engines are non-intrusive tools that allow components to remain independent, solely coordinating the exchange of data (Lawrence et al., Manuscript). There are significant overlaps between the technologies and they are often used in tandem (Lawrence et al., Manuscript). Based on the level of integration between the components, the coupling can be defined as either “tight” or “loose” (Goodall et al., 2011). Summarising, while all couplers have the same basic functions they differ in the level of component standardisation, the way the components are called and exchange data, and the degree to which they are integrated.

	The coupling technologies can be formally divided into: coupling libraries, coupling frameworks, and workflows (Lawrence et al., Manuscript, Dunlap et al., 2013). Libraries provide concrete solution fragments (Lawrence et al., Manuscript); they minimise the amount of code

	~~<p align="center">1</p> </div> <br clear="all"> <div>~~
	changes required to make a model couplable, typically allowing it to act as independent executable and merely to exchange data at appropriate locations and times (Dunlap et al., 2013). Frameworks use standard interfaces for communication with the components, which must comply with the interfaces' calling conventions (Dunlap et al., 2013). Consequently that components must be structured in accordance with a predefined architectural design (Dunlap et al., 2013). Workflow engines are non-intrusive tools that allow components to remain independent, solely coordinating the exchange of data (Lawrence et al., Manuscript). There are significant overlaps between the technologies and they are often used in tandem (Lawrence et al., Manuscript). Based on the level of integration between the components, the coupling can be defined as either “tight” or “loose” (Goodall et al., 2011). Summarising, while all couplers have the same basic functions they differ in the level of component standardisation, the way the components are called and exchange data, and the degree to which they are integrated.

	A large number of coupling technologies were developed up to date, which seemingly appears to be a redundant effort. However, this is not the case as different approaches address different, often conflicting demands, like: generality, flexibility, ease of use, accuracy, and performance (Jagers 2010).		A large number of coupling technologies were developed up to date, which seemingly appears to be a redundant effort. However, this is not the case as different approaches address different, often conflicting demands, like: generality, flexibility, ease of use, accuracy, and performance (Jagers 2010).

Dbk: Created page with "notoc {{OR/14/022}} ==Background== To make Integrated Modelling (IM) work, a way of passing data between models is required and to facilitate this, standards are necessary..."

2015-05-22T09:53:01Z

Created page with "__notoc__ {{OR/14/022}} ==Background== To make Integrated Modelling (IM) work, a way of passing data between models is required and to facilitate this, standards are necessary..."

New page

__notoc__
{{OR/14/022}}
==Background==
To make Integrated Modelling (IM) work, a way of passing data between models is required and to facilitate this, standards are necessary. Two sets of standards are required: data standards for one way, static transfer of data and model runtime standards for dynamic coupling. For the former, BGS environmental modellers use some basic standards: dxf, CSV, etc. However, it would be useful to identify internationally accepted standards that could be used for data exchange, particularly for gridded data. In terms of exchange of data during model runtime, the current standard and implementation used at BGS is OpenMI. OpenMI was designed with the solution of the problem posed by the Water Framework Directive, that of simulating catchment processes in a holistic manner. Therefore, the main aim of the OpenMI as it is currently implemented is for flexibility. It may not be appropriate in a high performance computing environment. Given that BGS’ requirements may change then it is necessary to identify and understand other standards or even approaches for linking models at runtime.

This report, therefore, focuses on the data standards for static and runtime coupling of numerical models used in the hydrological and atmospheric sciences. Included in this process are workflow engines, but approaches for other disciplines such as risk in the insurance industry and human health are not included.

==The need for couplers==
The need for interdisciplinary environmental modelling has become clear over the last decade as the evidence of the climate change has been growing stronger. Such modelling provides the means to study complex dynamics of the Earth system and thus aids finding ways to mitigate the impacts of the environmental change. In the year 2000, the Water Framework Directive was enacted, which recognised the need to implement integrated management strategies to address ever more rising and conflicting demands for water resources in a catchment. This problem is best addressed by adopting sound modelling approaches. Integrated modelling requires sharing and coupling models simulating different parts of the Earth system. The approach used to link such models is called “a coupler”. While a large number of different couplers are currently in use by scientists, their basic functions remain the same, namely: coordinating the execution of the coupled models and managing data transfer between them (Valcke et al., 2012).

The technologies used for coupling models vary in the level of “intrusiveness”, which can be defined as the amount of work required to make a component “couplable” (Lawrence et al., Manuscript). The coupling technologies can be divided into: monolithic, component-based, communication-based, and scheduled (Dunlap et al., 2013). The monolithic approach requires combining code from multiple models into one code (Dunlap et al., 2013). The component-based approach introduces the concept of standard interfaces. In this approach each model, called a “component”, has: an interface to communicate with other models, a structure in compliance with predefined criteria, and performs a distinct function (Dunlap et al., 2013, Lu 2011). In communication and scheduled approaches models are independent (Dunlap et al., 2013, Lu 2011). The communication approach requires embedding library calls within the model's code for sending and receiving data (Dunlap et al., 2013). In the scheduled approach the output from one model is used as an input to the next one, thus the models do not affect each other during the execution (Dunlap et al., 2013).

The coupling technologies can be formally divided into: coupling libraries, coupling frameworks, and workflows (Lawrence et al., Manuscript, Dunlap et al., 2013). Libraries provide concrete solution fragments (Lawrence et al., Manuscript); they minimise the amount of code

<p align="center">1</p> </div> <br clear="all"> <div>
changes required to make a model couplable, typically allowing it to act as independent executable and merely to exchange data at appropriate locations and times (Dunlap et al., 2013). Frameworks use standard interfaces for communication with the components, which must comply with the interfaces' calling conventions (Dunlap et al., 2013). Consequently that components must be structured in accordance with a predefined architectural design (Dunlap et al., 2013). Workflow engines are non-intrusive tools that allow components to remain independent, solely coordinating the exchange of data (Lawrence et al., Manuscript). There are significant overlaps between the technologies and they are often used in tandem (Lawrence et al., Manuscript). Based on the level of integration between the components, the coupling can be defined as either “tight” or “loose” (Goodall et al., 2011). Summarising, while all couplers have the same basic functions they differ in the level of component standardisation, the way the components are called and exchange data, and the degree to which they are integrated.

A large number of coupling technologies were developed up to date, which seemingly appears to be a redundant effort. However, this is not the case as different approaches address different, often conflicting demands, like: generality, flexibility, ease of use, accuracy, and performance (Jagers 2010).

==Coupler use cases and requirements gathered from BGS staff==
In 2010 the BGS produced the Dream Scoping study report (Giles et al., 2010), as part of the research for this report a wide range of BGS scientists, responsible for answering questions raised by clients, were asked what they required from a model linkage solution, below are a selection of those responses:

: ''“As a geologist focussed on the urban environment I want an environmental modelling platform to act as an effective communication tool, perhaps through visual representations of processes, so that others, including non-geologists, can better understand the model.”''

: ''“As a geoscience standards and property team member I want to be able to calculate the financial implications of varying sub-surface project options, for example 'where is the cheapest place to dig this tunnel?', so that our customers (& potential customers) understand the significance and benefits of sub-surface knowledge.”''

: ''“As a geophysicist I want an environmental modelling platform to handle high volumes of data traffic on a regular and ongoing basis, so that I can process real time data from the field or sensors, automatically model it and I & customers can view the results and identify trends.”''

: ''“As a flood analyst, I want to predict possible flood scenarios for the village over the next 24hours using various inputs such as rain fall, groundwater, water table levels, so that decision makers can be given the info necessary to decide whether the village should be evacuated.”''

At the time of capturing these use cases the imagined solution was referred to as an environmental modelling platform and opinions varied greatly on how much functionality would be delivered through the new platform and what existing components would be re- used. Despite significant differences in opinion it was possible to identify a common set of desirable attributes that any solution should exhibit.

==Commonly desirable model coupling technology attributes==
There is an almost bewildering choice of methodologies, technologies and tools available to integrated environmental modelling (IEM) practitioners, however there are some concepts which we regard as desirable.

The IEM technologies used by the BGS should incorporate the following attributes:
* Ability to link models in a modular way, rather than developing a single piece of code (model) that incorporates data manipulation and scientific logic we should encourage developers to separate out these functions so that they can be used in more than one scenario.
* Visual workflow builders open up the world of linked model development to users with little to no programming experience. Although care should be taken to ensure that any assessment of the performance of a linked model solution fully considers the impact of technological implementation as well as scientific logic, this becomes difficult when the user does not fully understand how a technology works behind the scenes.
* It should be simple to capture the metadata required to describe scientific models, the data they require and any data outputs generated, in order to support model discovery and provide guidance on how to use the model(s).
* Coupling technologies which exhibit a low degree of invasiveness tend to have less of a negative impact on the performance of existing models, extensive alterations can lead to code divergence and may adversely affect the original model design or purpose. In addition, alterations made for one technology can limit model re-use in alternative technologies.
* Technologies with significant community support provide potential users with a confidence that help is at hand should it be needed. The BGS should pay particular attention to the technologies favoured by communities who specialise in those areas of science we wish to integrate with.
* And finally, a ‘stable’ or clearly versioned technology provides the user with a certain degree of certainty that doesn’t exist with rapidly changing environments. Models and linked models can be assessed for their scientific value without the added confusion of a transient informatics platform. Although the technology should be stable, it is also desirable that there is an active, albeit separate, development path which helps to improve the technology in response to community needs.

==Structure of report ==
The following sections of the report describe in detail the dynamic (run-time) approaches for atmospheric and hydrological approaches (Section 2), which is followed by a summary of data standards for one-way, static transfer of data (Section 3). Section 4 compares the different approaches and the findings of the report are summarised in Section 5 along with providing recommendations for the next stage of work.

[[category:OR/14/022 Couplers for linking environmental models: Scoping study and potential next steps | 02]]