The Environmental Virtual Observatory (Pilot): A new ... · But, what is Cloud Computing (contd)? • Coulouris, Dollimore, Kindberg and Blair: – A cloud is defined as a set of

Realising the potential of environmental data, models and toolsRealising the potential of environmental data, models and tools

The Environmental Virtual Observatory (Pilot):A new vision for Catchment Science

PIs: B Emmett (CEH), R J Gurney (Reading), D Tetzlaff (Aberdeen), J E Freer(Bristol), A Mcdonald (Leeds), G Rees (CEH), W Buytaert (Imperial), G Blair(Lancasters), and P Haygarth (Lancaster)

CoIs:Keith Beven (Lancaster), John Bloomfield (BGS), Penny Johnes (Reading),Paul Quinn (Newcastle), Mark Macklin (Abserystwyth), Christopher Macleod(Macaulay), Sim Reaney (Durham), Marc Stutter (James Hutton Institute)

PDRAs: Marcia Tavares-Smith, (Lancaster), Mark Wilkinson (Newcastle), YehiaEl-Khatib (Lancaster), Claudia Vitolo (Imperial) Alastair Gemmell (Reading)Eleanor Mackay (Lancaster), Nicola Thomas (Aberystwyth), Nick Odoni(Bristol), and Keith Marshall (James uhtton institute)

Realising the potential of environmental data, models and tools

Challenges in catchment science

We are facing unprecedentedchallenges in the management of soiland water.

Our research increasingly also hasreal practical application.

However, many scientific andenvironmental challenges are verycross-disciplinary, and require use ofmultiple data, models andvisualisation tools across disciplines,organisations and topics.


Solutions?

Is there are a way of providing the‘wiring’ to help people access whatresources they need, be they ascientist, policy maker, industrialbody, regulator or public?

Resources would have to gobeyond data (not that useful tomost people)…. so models andtools (visualisation, decisionsupport and valuation)

©Shutterstock


What are the opportunities offered by newcloud technologies?

They offer a way to connect andintegrate our current fragmenteddata, models and knowledge.

The key to the cloud approach is therepresentation of everything as aservice, that is our data, models,visualisation tools and expertknowledge.

And then to view the cyber-infrastructure as a “cloud” of servicesout there that can be used andcombined in arbitrary ways, in this caseto solve environmental questions.


HOLY GRAIL OF INTEROPERABILITY

Realising the potential of environmental data, models and tools 6


Everything As a Service

[source: Zhang, Cheng, and Boutaba, 2010]

SalesForce.com


If you’re like me – you’re a bit lost now....

• Only one way of finding out – try it out on real exemplars:

– UK Natural Environment Research Council funded a 2 year pilot project

– Unusually, project driven by scientists, endusers and techies together

– Three scales to capture different issues and audiences (local, nationaland global)

– Deal with data, models and visualisations

• Make sure we ask people what they really want

– 17 PIs, 13 institutions, 12 Endusers on Project Advisory Board fromwater companies, policy, software companies, land managers etc

• Not afraid to fail but doing it early and iterate with an even wider groupof endusers

• Link across to other projects globally so as not to reinvent the wheel


Questions the EVO could help answer

ScientistsHow do we define which

model is better to select fromour model ensemble?

What kind of lanuage/tools touse to make models talk to

each other.

Water IndustryHow can we convert into saleable

product for UK plc?What is the whole impact of future

flooding?How can water security be assured?How can the industry carbon impact

be reduced?

PublicWill London run out of water?What is the state of the local

river?What are the options toprotect us from future

flooding?

RegulatorsHow can we reducemonitoring for same

information?Credible apportioning ofpollutant load between

industry, water, agriculture,other.

GovernmentAgencies

What models work?What policy works?

How do I do it for lessmoney?

EVO cloud


But this is not enough for the techies to go on…

• Which data needs to be assimilated? Static or live?

• Which models? How linked?

• How do you want to visualise outputs?

• Need to be realistic about what is possible so apps forphones are not a priority

• And needs to testand demonstrate fundamentals of thecloud (techies have their needs to!). Exemplars had to showadded value of cloud

Storyboards


Storyboard


User Login

What part of the

country do you

want to look at?Choose location

What are you

interested in?Choose topic

Submit

What level do you want to use?

Beginner

Intermediate

Expert

Submit

User logs in to portal

Issue: User either has to know how toregister and log in or must be shown.Needs to be clear instructions fromfront page

Issue: linking up between catchmentwebpages and EVOp needs thinkingabout

User selects location of area they areinterested in, may also need search boxby name or postcode.

User selects topic (only flood riskactive) – submit

Issue: Will this portal be for everyoneor vary by user or scale or use e.g. local,national?Issue: Who is responsible for interfacedevelopment?

User selects level at which they want tointeract with data. This screendifferentiates users and determineswhat accessibility is needed in the restof the portal.

Issue: Will this be defined by userprofile when registering? Or would thattake away functionality unnecessarily?

2. Entering the portal(Eden catchment example)


What are the land

uses in this area?

Slide land uses on

graph to changeGrazing, 80%

Crops, 10%

Woodland, 5%Urban, 5%

How well managed

is the land?

Use Slider

Bad

practice

Good

practice

SubmitSpecify management

Change

scenario

Ask a question

Upload info

What do you

think?

Return to

screen 3B

What does this mean?Animate prediction

Your Flood risk prediction #2

Chance of this area

flooding? Use Slider

What does

this mean?

Mouse over

graphVery HighVery low

What do you think?Calculate costs and

benefits of options

Impacts of Flooding #2 Tell me more

about:

Uncertainty in

these flood

estimates

Impact table

Impact Table Model run #1 Modelrun #2

Environmental X km2

Economic Loss offarmland £X,

Social X number ofhouses

Feedback form

Submit

Feedback on Model run #1

MyCatchment

Ask a question

Upload info

What do you

think?

Comments

Feedback on Model run #1….

Comments

Feedback on Model run #1….

5. Feedback and Iteration(Eden catchment example)

Feedback initially as a simple form whichcould be part of a blog but eventuallywould be able to tag model output mapsto georeference validation data withmap pins.

Issue: is this better within EVOframework or local community site?

Option to return to model input screen and rerun model with changed land use. Outputswould allow overlays on maps and additional columns in tables to allow comparisonbetween BAU and mitigation options.

Issue: Is this possible? Can the model runs be stored and looked at again?


BUT WHY WOULD ANYONE MAKETHEIR DATA, MODELS OR TOOLSAVAILABLE TO THE EVO CLOUD?


Security• Some datasets in the EVO will have access open to some but closed to others.

• How to handle access rights? Delegating the user’s credentials was the keyissue:

– User logs onto portal

– Portal then accesses secure data on user’s behalf (‘one hop’)

– Often Portal accesses WPS which in turn accesses secure data (‘two hops’)

• ‘Mash my data’ project solved these issues using OpenID and Proxycertificates. The user only has to sign on once and system keeps track of whatthey can do

– Similar scenario using it to run simple models such as in EVOp

• Security of data-altering also a major issue; Currently digital data is notadmissible in court e.g. for water quality.

Other projects found that security issues dictated thearchitecture of the whole system – so need to sort early.


Developments for interoperability

• Lessons from the CEHInformation Gateway(www.gateway.ceh.ac.uk)assimilating e.g. thePlynlimon legacy datasets,revealed a requirement fordictionaries to describe data.

• Links between locations, widearray of observations andanalyses need to be defined


Dictionary development


Gaining information from linked data


Current linked data map …



We will deliver a prototype portal

User ScenariosTopics Resources Services

Authentication

Inside CloudInside Cloud

Outside Cloud Outside Cloud

data

maps

models

Visualtools

Open Open


But …. most will be inactive with just our 3 exemplars live:Our aim is to show potential and thus drawdown next phase of funding

User ScenariosTopics Resources Services

Authentication

Inside CloudInside Cloud

Outside Cloud Outside Cloud

maps

Visualtools

Open Open

Imogendata

soil


International related Projects• CUAHSI

– Provides unified access to data, tools and models relating to hydrological research

– Very data-oriented, but could help in visualisation

• Critical zone observatories / Soiltrec– Major data assimlation effort, combining with hydrological models and visualisation tools - only project with major modelling

component we have come across

• INTAMAP– Provides real time visualisation of critical environmental variables

– Uses OGC WPS and R interpolation modules

– Developed UncertML

• NeOn– Provides support to semantic distributed applications

– Developed the NeOn toolkit to import and manage ontologies

• LIS– Distributes embarrassingly-parallel processes over a flexible computational facility

– Uses DODS to abstract remote data sources location and format

– Uses GrADS, Live Access Server (LAS), Ferret, Matlab, Excel, etc. for visualisation

• GIGAS– Focuses on the representation and access interoperation of 3 initiatives:

• GEOSS: climate, biodiversity, energy, weather, etc.

• GMES: land, sea and air

• INSPIRE: any spatial information2005 to 2015

• HUMBOLDT– Aims to demonstrate the benefits of harmonising spatial information using the standards of INSPIRE.


If the EVO is successful, what is the benefit forscientists?

• Enabling environmental researchers to concentrate onscience “Fewer wheels, more environmental science.”

• Not only those ‘in the know’ will be able to discover dataportals and information “Democratisation of science”

• Stop re-invention and repeated blind alleys e.g. re-

implementing models, file import and export routines etc.; End-users alsosuspect many contracts ‘reinvent’ work due to poor communication and lostknowledge

• Improved reputation of greater transparency andcontribution to societal needs


What would be wider implications…?

• The EVO would make our sciencewhich underpin management optionsand decisions to deal with our mosturgent environmental problems more:– Efficient

– Effective

– And transparent

• And provide tools to help managersidentify the best options for today andan array of likely futures



QUESTIONS?


But, what is Cloud Computing (contd)?

• Coulouris, Dollimore, Kindberg and Blair:– A cloud is defined as a set of Internet-based

application, storage and computing servicessufficient to support most users’ needs, thusenabling them to largely or totally dispensewith local data storage and applicationsoftware. The term also promotes a view ofeverything as a service from physical or virtualinfrastructure through to software, often paidfor on a per-usage basis rather than purchased

Documents

The Environmental Virtual Observatory (Pilot): A new ... · But, what is Cloud Computing (contd)? • Coulouris, Dollimore, Kindberg and Blair: – A cloud is defined as a set of