AI for Data-Driven Decisions in Water Management

© 2017 IBM Corporation

AI for Data-Driven Decisions in Water Managementhttps://sites.google.com/site/aiwatertutorial/

Biplav Srivastava, IBM Research

Sandeep Singh Sandha, PhD Student, UCLA (Ex-IBM)

Acknowledgements: Our colleagues at IBM Research and collaborators at various agencies.

Tutorial at AAAI 2017, San Francisco, USA, Feb 04, 2017

© 2017 IBM Corporation2

What to Expect: Tutorial Objectives

§ Background– The importance of water to life is unique. Humanity faces unprecedented water challenges.

– AI techniques have high potential to impact management of water resources– We want common citizens to make better decisions around water, and through them, motivate government and businesses

§ The aim of the tutorial is to– Make both early and experienced researchers aware of the water management area, its issues and opportunities

– Discuss available open water pollution data, both quantitative and qualitative in nature from a modality of sensors, that researchers can use to help a wide spectrum of decision makers, e.g., farmers, tourists, environmentalists, health officials, policy makers and businesses.

– Demonstrate usage of AI techniques to make real world impact that matters– Help create ecosystem for water informatics innovations


Acknowledgements

All our collaborators & partners over the last 3 years:§ Government agencies

– India: Center and States of Delhi and UP

§ Academia– India: IIT Roorkee, IIT BHU, IIT Madras, IIT Kharagpur– USA: Univ. of Chicago, University of Southern California, UCLA– Canada: York University

§ Non-profits– Center for Science and Technology (India)– 2030 Water Group– RiverKeeper, USA

§ Startup– Platypus, USA

§ IBM: Sukanya Randhawa, Supratik Guha, Theodore G van Kessel, Hendrik Hamann, Bharat Kumar, Jaikrishnan Hari, Sachin Gupta, KarthikVisweswariah, Anupam Saronwala, RashmiMittal, Deepak Turaga, Kush Varshney

For discussions, ideas and contributions. Apologies to anyone unintentionally missed. Material gratefully taken from multiple sources. Apologies if any citation is unintentionally missed.


Outline

§ Motivating Examples

§ Water: Basics and Problem

§ Towards Solution– Data & Infrastructure

• Data Sources• Existing Mobile Apps • BlueWater and GangaWatch

– Experience Working in the Field– Usage Scenario

• Inspection• Tourism Impact

§ Discussion– AI Research Issues– Water Data Standards– Call for Action

4


Motivating Examples


Many Views of Water

Buenos Aires, Argentina, July 2015

Chennai, India, April 2015


[India] Ganga – Local Ground Situation @ Varanasi (Assi/ Tulsi Ghats) + Patna

Photos of/ at Assi/ Tulsi Ghat, Varanasi on 25 March 2015 during 1700-1800 Hrs

Assi Ghat post recent cleanup Bathing on Tulsi Ghat

A nullah draining into GangaA manual powered boat

Photos at Gandhi Ghat, Patna on 18 March 2015during 1700-1800 Hrs

Common scene around Indian water bodies


[India] Ganga – Local Ground Situation Near Haridwar

Photos at different spots along Ganga at Haridwar-Rishikesh’s 70Km stretch during Feb-Apr 2016


Decision Example –River Water Pollution

§ Value – To individuals, businesses, government institutions– Individual Use Example – Can I take a bath? Will it cause me dysentery? Can I drink this water? Can I use this water for irrigating my kitchen garden? Can I eat the fish I catch?

– Business Use Examples – How should govt spend money on sewage treatment for maximum disease reduction? If an industry is being set-up, which site is best among feasible alternatives with least impact and latest technology?

§ Data – Quantitative as well as qualitative– Dissolved oxygen, – pH, – … 30+ measurable quantities of interest

§ Access –– Today, little, and that too in water technical jargon– In pdf documents, website

Key Idea: Can we make insights available when needed and help people make better decisions?


GangaWatch Video Demo

Video at: https://youtu.be/MbVvVGsZoTo

GangaWatch App on Android store, https://play.google.com/store/apps/details?id=com.ibm.research.gangawatch

GangaWatch blog - https://www.linkedin.com/pulse/ganga-watch-app-using-water-data-every-day-decisions-srivastava


Initiative: UK

§ Summary– A web-based interface specific to bathing atmajor water-bodies

§ Reference– https://environment.data.gov.uk/bwq/profiles/


Initiative: New South Wales, Australia

§ Summary– Web and Mobile based tools to visualize water availability in river bodies in New South Wales

§ Reference– http://waterinfo.nsw.gov.au/– http://realtimedata.water.nsw.gov.au/water.stm


Initiative: US Geological Survey§ Summary– Collects water data from different locations, makes available online along with web services to access them programmatically

– Interpretation of goodness for a purpose left to user;; Clear Water Act and state laws impact interpretation

§ Reference– https://www2.usgs.gov/water/, https://waterwatch.usgs.gov/wqwatch/ https://www.epa.gov/laws-regulations/summary-clean-water-act


Initiative: Jefferson Project at Lake George

§ Summary– Joint collaboration between academia and industry, project tries to collect large-scale water data and do hydrological modeling;; help promote conservation

– Example of outcome: Zooplankton Rapidly Evolve Tolerance to Road Salt, https://www.rdmag.com/news/2017/01/zooplankton-rapidly-evolve-tolerance-road-salt

§ Reference– https://fundforlakegeorge.org/JeffersonProject


Initiative: RiverKeeper for Hudson River

§ Summary– Impact on recreation activities due to human waste contamination

§ Reference– http://www.riverkeeper.org/water-quality/hudson-river/


Initiative: Sewage Processing in Spain

§ Summary– Optimizing operation of sewage processing regularly rather than statically / seasonally.

– In city of 2 million people in Spain, an operation optimization system showed a dramatic 13.5 percent general reduction in the plant’s electricity consumption, a 14 percent reduction in the amount of chemicals needed to remove phosphorus from the water, and a 17 percent reduction in sludge production.

§ References– https://www.ibm.com/blogs/research/2016/08/used-iot-data-optimize-wastewater-treatment/

– Operational optimization of wastewater treatment plants: a CMDP based decomposition approach, Zadorojniy, A., Shwartz, A., Wasserkrug, S. et al. Ann Oper Res (2016). At: http://link.springer.com/article/10.1007/s10479-016-2146-z


Analytics: Potential Use CasesS.No.

Stakeholder

Use case Data Analyticaltechniques

1 IT Identifying and removing outliers, data validation

Sensor data Data mining (outlierdetection)

2 Individual Which bathing site to use? Sensor data, ghat data Rule-based decision support

3 Individual/ Economy

What crops can I grow that will flourish in available water?

Sensor data, crop data Distributed data integration, co-relation

4 Institution Determine trends/anomalies in pollution levels

Sensor data, weather data

Time series analysis,anomaly detection

5 Institution Attribute source of pollution at a location

Sensor data, demographics, industry data

Physical modeling, inversion

6 Institution Sewage treatment strategy and operational planning

Sensor data, demographics, STP data

Multi-objectiveoptimization

7 Institution Promoting wildlife/ dolphins Sensor data, wildlife data Rule-based decision support

8 Institution Using waterways for commercial shipping

Sensor data, commercial shipping routes

Optimization, planning

17


Water Basics and Problem


Water Cycle (aka Hydrological Cycle)

Source: Economist, May 20, 2010


Fresh Water: Supply and Demand

Source: Economist, May 20, 2010

Supply Demand


Water Usage Trivia


Fresh Water Stress: Spatial Distribution

Freshwater stress will affect both developed and developing nations. Billions will be affected.


Water Challenges Summary

§ Increasing demand due to– Population– Changing water-intensive lifestyle– Industrial growth

§ Shrinking supplies– Erratic rains due to climate change– Sewage / effluent increase

§ Poor management– Below cost, unsustainable, pricing– Delayed or neglected maintenance

Water is the next flash point for wars


Better Information Flow is Critical for Better Water Flow

“One barrier to better management of water resources is simply lack of data — where the water is, where it's going, how much is being used and for what purposes, how much might be saved by doing things differently. In this way, the water problem is largely an information problem. The information we can assemble has a huge bearing on how we cope with a world at peak water.”Source:Wired Magazine, “Peak Water: Aquifers and Rivers Are Running Dry. How Three Regions Are Coping”, Matthew Power, April 21st, 2008

The nature of water management must rapidly evolve

From To

Manual Data Collection Automated SensingManaging Collaboratively

Intermittent Measurement Real-Time MeasurementMultiple Data Sets Data Integration“Guesstimation” Tools Modeled Decision SupportCommodity Pricing Value PricingTactical Problem Solving Strategic Risk Management

Managing in Isolation


Decisions Within Water Life Cycle

11

1

Water Sources• How much water is available?• What is the quality of the water?• How secure is the water?• How is the water changing over time?• How is/should the water be mixed?• Do we comply with regulations/water rights?

1

2

2 2

Modeling and forecasts•Feeds to modeling (e.g. SWIM)

• Meteorological models• Hydrological Models

• Groundwater protection• Feeds to storm water management• Alerts to municipalities/first responders

22

2 2

2

2

33

33

3 3

3

3

3

Metering Analytics• How is the water being used?• When is the water being used?• When patterns change dramatically, what to do?• When the sum of the parts is less than the whole.. Where is the water going?

3Combined Sewer Overflow• Was this truly an event?• How large was it?• What was the composition?• Can the existing capacity be used better?• How do I notify downstream entities?• How do I coordinate upstream agencies?• How can we backtrack to owners?

4

4

4

4

4

4

4

4

Environmental Analytics• How is the environment changing?• How do we notify interested parties as it changes?• How does the entire system react to changes?• How can we predict changes?

5

55

5

5

5

5

5

5

Advanced Water Management Capabilities


Towards Solution: Data & Infrastructure


Water Pollution Sensing

§ Method 1: Sample collection and lab-testing– Accurate when done well;; rich historical data which is under-utilized;; open data technologies can make them accessible

– Time-consuming, costly and thus feasible for a few places only at a time– Only quantitative– Science: lab tests, sample collection

§ Method 2: Real-time sensing– Timely, inexpensive– Some important parameters are NOT feasible but can be inferred (e.g., BOD, FC)– Only quantitative– Science: how to deploy sensors and analyze data

§ Method 3: Crowd-sourcing– Timely, inexpensive– Only qualitative assessment– Practical for India with people and mobiles– Science: Combining qualitative and quantitative data


Quantitative Sensing ScopeDimension Yamuna | Hindon| GangaScenario focus General, Agriculture

Real-time measurement DO, pH, conductivity, turbidity

Lab / samples BOD, COD, FCC

Sensing COTS sensors, Machine learning, In-lab test

Data ingestion Bluemix cloud, Cloudantdatabase

Primary§ Temp§ ORP§ D.O§ EC§ Turbidity§ Pressure§ Nitrate § GPS Lat§ GPS Long

Secondary

• Resistivity• TDS• Salinity• SeaWater Sigma

Sensor Measures

§ Microcontroller§ Sensor probes§ Communicator (Shields)

§ Data Storage (Server)


Water Qualitative Data Via Crowdsourcing –NeerBandhu App

Data at http://nalanda.haifa.il.ibm.com/naturetrack/visualization.php


Creek Watch – Crowd Sourced Water Information Collection

As on 14 Oct 2014


Location: http://creekwatch.researchlabs.ibm.com/call_table.php

~3120 data points in 4 years from around the world

As on 14 Oct 2014


Gaps Filled by Our Approach

§ High spatial and temporal resolution (real-time)– Current data are at low resolution of few places and limited time points;; limits usage in applications

– Use floating platform and real-time sensor to collect GPS-enabled data

– Use location to re-create water body condition

§ New source of data (qualitative;; crowd-sourcing) § Fusion of historic and new real-time data on single platform with safety levels and purpose

§ Future: contextualize quantitative data with qualitative inputs for data validation and stakeholders buy-in


Demo: GangaWatch (1/2)

Data Covering Ganga BasinFine-grained Geo-tagged Data from a Real TimeRun on Yamuna


Demo: GangaWatch (2/2)


Blue Water Architecture: Water Data

Data (Multiple Sources)

• Stationary Stations• Labs• Mobile Stations• Historical Data• ….

Applications

Web Service

Authenticated:• Data Upload

(Excel, CSV)• Meta Data

Upload• Auto Meta Data

generation

NoSQL Database: Cloudant

Stores: Data (Semi-Structured Data)Meta Data (Places, Limits)

IBM Cloud : Bluemix

Data Query Support

Spatio-temporal QueriesMeta Data Based Queries

https://bluewater.mybluemix.net/

Rest API

AuthenticatedMultiple queries are supported

1 2

3

4

56


BlueWater In-Depth: Web Service

Authenticated Data Uploadhttps://bluewater.mybluemix.net

Manual Places Definitions

Limits on Water Quality

Water Quality Data of Parameters in CSV

Water Quality Data of Parameters in Excel


BlueWater In-Depth: REST API

License Key Generation:https://bluewater.mybluemix.net/license.jsp

Add few basic details and get license online.

Supported queries§Spatial query§Temporal query§Mixed query§Meta data query



API URL: https://bluewater.mybluemix.net/query (All results are returned in json)Returns the info about data: Parameters, Units, statistics of data

Spatial queryInputs: Latitude, Longitude, Range and LicenseReturns: Data in Json

Example:https://bluewater.mybluemix.net/query?Latitude=28.66501&Longitude=77.2393&Range=10&Lic=XXX

Data Returned:

"source":"live_sensor_data","lat":28.66505,"lng":77.23923,"unix_timestamp_creation":1485297242,"userid":"ibmadmin","date":"2015/12/16","time":"12:02:48","datetime":1450267368,"Water_Temperature":15.94,"pH":7.82, ……

(Lat,Lng)

R

Haversine



Temporal queryInputs: Date_1, Date_2 and License

Time_1, Time_2 and License

Returns: Data in JsonExample:https://bluewater.mybluemix.net/query?Date_1=2015/12/16&Date_2=2015/12/18&Lic=XXXhttps://bluewater.mybluemix.net/query?Time_1=1450267414&Time_2=1450267426&Lic=XXX

Mixed queriesExamples:

https://bluewater.mybluemix.net/query?Latitude=28.66501&Longitude=77.2393&Range=1000&Time_1=1450267414&Time_2=1450267426&Lic=XXXhttps://bluewater.mybluemix.net/query?Latitude=28.66501&Longitude=77.2393&Range=10&Date_1=2015/12/16&Date_2=2015/12/16&Lic=XXX

Date = YYYY/MM/DD Time = Unix Timestamp(UTC)



Meta Data queryInputs: Places parameters and License

Limits and License

Returns: Data in Json Example:https://gw-ser1.mybluemix.net/query?Places-By-User=UserID&Place-Type=type&Lic=XXXhttps://gw-ser1.mybluemix.net/query?Limits-By-User=UserID&Limit-Type=drinking&Lic=XXX

Rest API Demo (Demo Usage Code in Java)https://github.com/sandeep-iitr/BlueWater_REST_API_DEMO


BlueWater In-Depth: GangaWatch Using REST API

Rest API

AuthenticatedMultiple queries are supported


Experience Working in the Field

Water-bodies of focus: (1) Hindon, sub-tributary Yamuna, tributary of Ganga(2) Yamuna, tributary of Ganga(3) Ganga


Experience Working in the Field - Hindon

River: Hindon (sub-tributary Yamuna, tributary of Ganga)


Hindon, Near Meerut, India (Sep 2015)

Places on HindonRiver

1. Kinoni Village (K)2. Barnawa (Ba)3. Budhana Road (Bu)4. Budhana (Bu-City)5. Titavi, Muzaffarnagar (T)6. Saharanpur

1

2

3

4

5


Hindonon the Ground

Kinoni Village (K)

Barnawa (Ba)

1

2

3

4

5


Data Around Hindon

46

Historical Qualitative Collected


Location Findings

K Ba Bu Bu-City TDepth 3 feet 1-2 feet 5-6 feet 1 feet 3 feetBoat navigation

100 m 100 m 200 m 10-20 m 50 m

Water flow Medium Low Medium Low FastVisible Trash

Low Low Medium High Low

Distance to major highway

Far Near Near Near Near


Insights from Hindon

§ Boat cannot go at many places– Paddle boat may be more useful than powered boat– At one stretch, boat can go for 100m max at most places visited

§ Mobile data collection was done and useful with NeerBandhu;; GPRS signal strong during the whole trip

§ Diseases prevalent– Humans: Cancer, gastro, infertility, skin diseases– Animals: infertility, sudden death

• Current water usage: Bathing cattle, irrigating fields, drinking by buffalo§ Impact of sensed data/ use cases: using river water data for– Plantation of trees along the river bed– Distribution of water filtration systems or setup of overhead tanks by government in villages (Note: current data may itself be useful)

– Spreading awareness about river water usage for vegetables has lead to change


River: Yamuna (tributary of Ganga)

Experience Working in the Field - Yamuna

Reference:

A multi-sensor process for in-situ monitoring of water pollution in rivers or lakes for high-resolution quantitative and qualitative water quality dataSukanya Randhawa, Sandeep S Sandha and Biplav Srivastava, 14th IEEE/IFIP International Conference on Embedded and Ubiquitous Computing (EUC 2016), August 2016.


Real-Time Sensor Deployment


Day 1 -‐ multiple anchoring approaches for real-‐time sensor on another day (16 Dec) in 2-‐3 km stretch

16-‐Dec-‐15

Location Name DescriptionSample -‐collected Sample -‐ testing Sensor @site

Realtime (Stretch) Neer Bandhu

1Point 1 [A] Nigambodh, in waterY

Y (ph, DO, Temp, Turb, Cond, BOD, FCC) Y Y

2Point 2 [B] Y Y3Point 3 [C] ITO bridge Y Y Y4Point 4 [D] Y Y

5Pointe 5 [E] YY (ph, DO, Temp, Turb, Cond) Y

6Point 6 Moving (7-‐8 Kmph) Y7Point 7 Moving (10 Kmph) Y Y

8Point 8 Drain YY (ph, DO, Temp, Turb, Cond) Y Y

9Point 9 With Ted buoy Y


Dec 16Example Run

16/12/15 13:59:34

16/12/15 13:46:50

• ~12 minute downstream travel

• 765 data points


Turbidity in Yamuna – measured on 16th Dec, 2015Data min: 56.7Data max: 127Gradient: Default

56

91

127TurbidityFNU


Day 2 -‐ Covered ~7-‐8 km one-‐way on one of the days(18 Dec) roughly covering 33 % of the navigable stretch of Yamuna in Delhi (22 km one-‐way).

18-‐Dec-‐15

Location Name DescriptionSample -‐collected Sample -‐ testing Sensor @site

Realtime (Stretch) Neer Bandhu

1Point 21 [AA] Nigambodh, in waterY

Y (ph, DO, Temp, Turb, Cond, BOD, FCC) Y Y

2Point 22 [AB] Past rope (ISBT) Y Y3Point 23 [AC] 2nd rope Y Y

4Point 24 [AD] Drain YY (ph, DO, Temp, Turb, Cond) Y Y Y

5Pointe 25 [AE] Drain Y Y6Point26 [AF] Drain, gurudwara Y Y

7Point 27 [AG] Wazirabad bridge YY (ph, DO, Temp, Turb, Cond) Y Y Y

8Point 28 [AH] Majnu ka tila, greenery Y Y

9Point 29 [AI] 1st rope, ISBT Y


Dec 18Example Run

2015/12/18,12:13:45

2015/12/18,12:51:37

• ~38 minute upstream travel

• 2273 data points


Turbidity in Yamuna – measured on 18th Dec, 2015Data min: 50Data max: 144.4Gradient: Default

50

97

144TurbidityFNU

56

91

127TurbidityFNU

Reference:Turbidity of 16-Dec


Turbidity and Conductivity in Yamuna – measured on 18th Dec, 2015

50

97

144TurbidityFNU

1268

1424

1582ConductivityµS cm


Lab Samples and Traditional Testing


16/12/2016 18/12/2016Temp(°C) 15.93 15.34pH 7.82 7.81ORP(mV) -182 -86.4D.O(mg/L) 3.76 3.53EC (µS/cm) 1604 1279Turbidity (F.N.U) 84.25 66.9BOD (mg/L) 46 28.2Fecal Coliform(No./100 mL) 430 210

Change in parameters measured for two different days

Sensor

Lab


NB Qualitative Data

http://nalanda.haifa.il.ibm.com/naturetrack/visualization.php


Correlating RT Sensor and Crowd Data to GetVerifiable Data!


Towards Solution: Usage Scenario


Towards Solution:

Usage Scenario – Inspection

Reference:

Protecting the NECTAR of the Ganga River Through Game-Theoretic Factory Inspections,Benjamin Ford, Matthew Brown, Amulya Yadav, Amandeep Singh, AruneshSinha, BiplavSrivastava, Christopher Kiekintveld, Milind Tambe,14th International Conference on Practical Applications of Agents and Multi-Agent Systems, Sevilla, Spain, June 1-3, 2016


Art of Possible

Tannery Example: Kanpur, India


Background of Leather Tanning Problem

• > 700 tanneries in Kanpur– Employing > 100,000 people– Bringing > USD 1B revenue

• Discharge water after leather processing to river or Sewage treatment plants (STPs)– Requirement

• Must have their own treatment facility• Or, have at least chrome recovery unit

– But don’t implement due to costs which is a burden to main operations• Installation• Operations : electricity, manpower, technology upgrade, …

– State pollution board is supposed to do inspections to enforce but doesn’t perform effectively

• Government’s STPs do not process chrome, the main pollutant • Knee-jerk reaction: 98 tanneries banned in Feb 2016 by National Green Tribunal;; more threatened


Flow Chart of Tanning

Source: home.iitk.ac.in/~sgupta/tannery_report.pdf


Water Pollutant Standards for Tanning

Pollutant standards to maintain depends on whether effluent discharged directly to river or to STP in-drain

•To river directly– Chrome: < 1 mg/l– Sulphide: < 2 mg/l– Suspended solids: < 100 mg/l– Ph: 6.5 – 9– COD: < 250 mg/l

•To STP (after which, STP will process) – Suspended solids: < 600 mg/l– Ph: 6.5 – 9

• Current state– Chrome: 45 mg / l– Other heavy metals are also at alarming level: arsenic, mercury, nickel


Consequences of Contaminated Water in Kanpur

§ Poor public health in Kanpur and downstream

§ Affects poorest the most and government eventually– Rural people in India (70%) spend at least Rs.100 each year for the treatment of water/sanitation-related diseases. Which is approximately same as Central Govt’s Health budget (Rs 6700 crore;; $US 1.1B);; doesn’t factor the costs to urban India (30%)

– Similar implications likely for Kanpur area too

§ Stresses future water supply: lowering ground water table§ Slows future industrial growth and thus economy


NECTAR: Nirikshana for Enforcing Compliance for Toxic wastewater Abatement and Reduction.

Setting• Attackers

• M sites with N factory units each• When inspection at a site happens, all units know

• Defenders• Inspectors base office is fixed• Inspection team consists of

• Environment Inspectors• Security personnel• Transport provider / drivers

• Inspection team starts and ends at their office

• Security and transport can vary daily

• Objective• Create daily inspection plan which minimizes violation over a time period

Joint work with USC, USA


Demo: NECTARhttp://teamcore.usc.edu/people/benjamin/ganga/gangaDemonstration.htm

"Very promising approach. Use of decoys and data-driven random were not known in the inspection community where it was known that random could help. Surprise elements of decoys and variable fines provide new factors for compliance. The data from drone monitoring can help improve the plans significantly as future work."

Dr. Venkatraman Rajagopalan, IAS Ex-Secretary, Ministry of Environment, Forests and Climate Change, and Ex-Chairman, Central Pollution Control Board, India


AI in the Scenario

§ Inspection generation a green security game - StackelbergSecurity Game

§ Solve as MDP to generate inspection plan and schedule§ Explain plan and what-if questions using simulation and rules

§ Advanced– Utility-driven planning - Use pollution data to update inspection objective and replan

– Mechanism design - Determine optimal fine strategy


Towards Solution:

Usage Scenario – Tourism Impact

Reference:

An Open, Multi-Sensor, Dataset of Water Pollution of Ganga Basin and its Application to Understand Impact of Large Religious Gathering,B Srivastava, S Sandha, V Raychoudhury, S Randhawa, V Kapoor, A AgrawalarXiv preprint arXiv:1612.05626


Use-Case: Understand Impact of a Large-Scale Religious cum Tourism Event

§ Haridwar Ardh Khumbh Mela 2016– January 1, 2016 to April 30, 2016– Over 100 millions attended;; Many took dip in river at select spots

– Major bath sub-events during the period have high burst of visitors

§ Question– How much does human activity impact river?– Where is the impact highest? Of what kind?

Sources:1. https://en.wikipedia.org/wiki/Kumbh_Mela2. http://www.kumbhamela.net/kumbha-mela-haridwar.html2. http://www.thegreatananda.com/ardh-kumbh-mela-2016-haridwar/

Date (2016) Day Main Bathing Event (Snan)

14th January Thursday Makar Sankranti12th February Friday Vasant Panchami22nd February Monday Magh Purnima7th March Monday Mahashivratri7th April Thursday Chaitra Amavasya

8th April Friday Chaitra Shukla Pratipada

14th April Thursday Mesha Sankranti15th April Friday Ram Navami

22nd April Friday Chaitra shuklaPurnima


Ardh Kumbh 2016, Haridwar

Territorial Bird’s Eye View: ~76 KM (Road Distance)


Turbidity Variations Feb 27-28, 2016

Turbidity values at different places (places marked red have turbidity value above the drinking range, places marked blues ha turbidity value in range of drinking water)


Data Collection Points around Har-ki-pauri, HaridwarFeb 27-28, 2016

Carrying sensor on a buoy for long stretch was not possible due to water speed.

45+ places from Rishikesh to Ganga Canal (Roorkee) (75+ KM)


Pollution on Major Bath Day around Har-ki-pauri, HaridwarMarch 7, 2016

Turbidity values at different places (places marked red have turbidity value above the drinking range, places marked blues ha turbidity value in range of drinking water)



AI in the Scenario

§ Data cleaning, normalization, missing values§ Quantitative to qualitative data conversation

§ Water Quality Index

§ Aggregate Binary Clustering using parameters with opposite polarities (e.g., Dissolved Oxygen, Turbidity), interval functions (e.g., pH)

§ Advanced– Generating consolidated qualitative assessment across multiple parameters

– Explaining and validating assessment


Discussion


Recap: Deliver Value From Water (Pollution) Data

§ Government for business decisions– Source attribution, inspection– Sewage treatment– Public Health

§ Individuals for personal decisions– Bathing (Religious, Lifestyle)– Recreation– Community practices


AI Research Issues

§ Sensing– Deciding sensors to use (multi-objective optimization)– How to sense cost-effectively? (Quantitative sensing)

• Install sensors• Ensure sensor up-keep, inspections• Decide sampling rate for sensors

– How to involve people-as-sensors? (Qualitative sensing)• Use people as inspectors (increase resources for defense)• Mobilization when needed on short notice• Devising incentives for contribution


AI Research Issues

§ Interconnection– Within water: quantitative and qualitative estimates;; relation between fresh and sewage water

– Across domains: energy implications on water management, physical safety, waste water treatment

§ Analytics– Decision-support (optimization, planning, scheduling) for organizing large-scale human activities

– Optimizing short-term and long-term investments (capex/ opex) for maximizing overall-value from invested water assets

– Pricing to incentivize water conservation and behavioral change


Water Data Standards§ A dis-organized collection of regulations varying by purpose (e.g., drinking, agriculture) and regions (country, state).

§ Conflicts may occur;; some data may have privacy considerations (e.g., flow volume);; standards often mixed with testing methods

§ WHO: water quality standards (from health perspective). At: http://www.who.int/water_sanitation_health/publications/whoiwa/en/

§ US: Environment Protection Agency’s water quality standards– https://www.epa.gov/wqs-tech– http://water.epa.gov/scitech/swguidance/standards/– https://www.epa.gov/wqs-tech/state-specific-water-quality-standards-effective-under-clean-water-act-cwa

§ Europe: http://ec.europa.eu/environment/water/index_en.htm

§ India: Central Pollution Control Board, State Pollution Boards– http://www.cpcb.nic.in/Water_Quality_Criteria.php– CPCB guidelines for real-time data, http://www.cpcb.nic.in/FinalGuidelinse.pdf– CPCB's list of pollutants, http://cpcb.nic.in/list_of_parameters.pdf


Call for Action

§ Use water data from BlueWater and find new insights– Use APIs– Build apps and/ or reuse GangaWatch app code

§ Collect and contribute your own data– APIs exist to upload;; registration need

§ Focus on a water use-cases and look at how you can formulate a basic problem;; solve them– Fishing– Water-borne public health– …


Conclusion

§ We highlighted the importance of water and gave a snapshot of potential for water informatics

§ Presented the BlueWater platform and GangaWatch app§ Shared field-experience collecting and using data§ Demonstrated use-cases of providing decision-support using AI techniques in water context

§ Tutorial can serve as a resource for others to contribute


References / Resources

§ Peter Gleick et al, The World's Water, Volume 8, The Biennial Report on Freshwater Resources, 2014

§ IEEE Spectrum Special Report: Water vs Energy (June 2010), http://spectrum.ieee.org/static/special-report-water-vs-energy

§ Economist Special Report (May 22, 2010), For Want of a Drink: Special Report on Water

§ World Bank– InvestIng In Water Infrastructure:Capital, Operations and Maintenance, Diego J. Rodriguez, Caroline van den Berg and Amanda

McMahon, 2012, At: http://water.worldbank.org/sites/water.worldbank.org/files/publication/water-investing-water-infrastructure-capital-operations-maintenance.pdf

– High and Dry: Climate Change, Water, and the Economy, http://www.worldbank.org/en/topic/water/publication/high-and-dry-climate-change-water-and-the-economy

– World Bank India Ground Water Report, Mar 2010

§ Binayak Ray: Water – The Looming Crisis in India, 2008, ISBN-13: 978-0739126028

§ IBM– Smarter Water Management Thought Leadership , http://www.ibm.com/smarterplanet/water– Smarter Water Management Solutions Home Page, http://www.ibm.com/green/water– IBM GIO Report on Oceans and Water, http://www.ibm.com/ibm/gio/water.html– IBM Water Management Pains Summary Report, http://www-935.ibm.com/services/us/gbs/bus/pdf/ibm-water-pains-report-jan09.pdf


Blue Water ResourcesApps / tools§ GangaWatch app for water pollution information on Google playstore. See description here in the blog.(Jan 2016

§ NeerBandhu (Water Friend) app on Google playstore. See description here in the blog.(Nov 2015)

Data§ From Blue Water/ GangaWatch, REST APIs are at: http://researcher.watson.ibm.com/researcher/view_group_subpage.php?id=7142 .

§ From Neer Bandhu (Water Friend), also available from the mobile app. Find at: http://nalanda.haifa.il.ibm.com/naturetrack/visualization.php

§ From Creek Watch, also available from web site. Find http://creekwatch.researchlabs.ibm.com/call_table.php

Blogs§ Ganga Watch and Blue Water Revisited – Collected Data is Now Externally Available, As-is,Aug 2016.

§ Ganga Watch – An App for Using Water Data in Every Day Decisions, Jan 2016.

§ Two Days on Yamuna – Collecting Water Pollution Data is Both Simple and Complex, Dec 2015.

§ Neer Bandhu, for Water Friends, Nov 2015.§ Can Game Theory Help Clean the Ganga? Randomized Inspection is a Case in Point, Benjamin Ford, BiplavSrivastava*, Milind Tambe, Sep 2015. At data.gov.in'senvivonmentcommunity.

Papers§ An Open, Multi-Sensor, Dataset of Water Pollution of Ganga Basin and its

Application to Understand Impact of Large Religious Gathering, Biplav Srivastava, Sandeep Sandha, Vaskar Raychoudhury, SukanyaRandhawa, Viral Kapoor, Anmol AgrawalOn Arxiv at: http://arxiv.org/abs/1612.05626, December, 2016.

§ The GangaWatch Mobile App to Enable Usage of Water Data in Every Day Decisions Integrating Historical and Real-time Sensing Data,Sandeep Sandha, Biplav Srivastava, Sukanya Randhawa,Demo paper on Arxiv at: http://arxiv.org/abs/1701.08212, January 2017.

§ A multi-sensor process for in-situ monitoring of water pollution in rivers or lakes for high-resolution quantitative and qualitative water quality dataSukanya Randhawa, Sandeep S Sandha and Biplav Srivastava, 14th IEEE/IFIP International Conference on Embedded and Ubiquitous Computing (EUC 2016), August 2016.

§ The GangaWatch App and BlueWater Platform to Enable Usage of Water Data in India in Everyday Decisions Integrating Historical and Real-time Sensing Data,Sandeep S. Sandha, Sukanya Randhawa, and Biplav Srivastava,Data Flow: Grand Challenges in Water Systems Modeling, Data Management, and Integration, Louisiana State University, Baton Rouge, May 9-10, 2016.

§ Protecting the Nectar of the Ganga River through Game-Theoretic Factory Inspections, B. Ford, A. Yadav, A. Singh, M. Brown, A. Sinha, B. Srivastava, C. Kiekintveld, M. Tambe,14th International Conference on Practical Applications of Agents and Multi-Agent Systems, Sevilla, Spain, June 1-3, 2016.

§ Blue Water: A Common Platform to Put Water Quality Data in India to Productive Use by Integrating Historical and Real-time Sensing Data, Sandeep S Sandha, Sukanya Randhawa and Biplav Srivastava, IBM Research Report RI15002, 2015.