© 2017 IBM Corporation
AI for Data-Driven Decisions in Water Managementhttps://sites.google.com/site/aiwatertutorial/
Biplav Srivastava, IBM Research
Sandeep Singh Sandha, PhD Student, UCLA (Ex-IBM)
Acknowledgements: Our colleagues at IBM Research and collaborators at various agencies.
Tutorial at AAAI 2017, San Francisco, USA, Feb 04, 2017
© 2017 IBM Corporation2
What to Expect: Tutorial Objectives
§ Background– The importance of water to life is unique. Humanity faces unprecedented water challenges.
– AI techniques have high potential to impact management of water resources– We want common citizens to make better decisions around water, and through them, motivate government and businesses
§ The aim of the tutorial is to– Make both early and experienced researchers aware of the water management area, its issues and opportunities
– Discuss available open water pollution data, both quantitative and qualitative in nature from a modality of sensors, that researchers can use to help a wide spectrum of decision makers, e.g., farmers, tourists, environmentalists, health officials, policy makers and businesses.
– Demonstrate usage of AI techniques to make real world impact that matters– Help create ecosystem for water informatics innovations
© 2017 IBM Corporation3
Acknowledgements
All our collaborators & partners over the last 3 years:§ Government agencies
– India: Center and States of Delhi and UP
§ Academia– India: IIT Roorkee, IIT BHU, IIT Madras, IIT Kharagpur– USA: Univ. of Chicago, University of Southern California, UCLA– Canada: York University
§ Non-profits– Center for Science and Technology (India)– 2030 Water Group– RiverKeeper, USA
§ Startup– Platypus, USA
§ IBM: Sukanya Randhawa, Supratik Guha, Theodore G van Kessel, Hendrik Hamann, Bharat Kumar, Jaikrishnan Hari, Sachin Gupta, KarthikVisweswariah, Anupam Saronwala, RashmiMittal, Deepak Turaga, Kush Varshney
For discussions, ideas and contributions. Apologies to anyone unintentionally missed. Material gratefully taken from multiple sources. Apologies if any citation is unintentionally missed.
© 2017 IBM Corporation4
Outline
§ Motivating Examples
§ Water: Basics and Problem
§ Towards Solution– Data & Infrastructure
• Data Sources• Existing Mobile Apps • BlueWater and GangaWatch
– Experience Working in the Field– Usage Scenario
• Inspection• Tourism Impact
§ Discussion– AI Research Issues– Water Data Standards– Call for Action
4
© 2017 IBM Corporation5
Motivating Examples
© 2017 IBM Corporation6
Many Views of Water
Buenos Aires, Argentina, July 2015
Chennai, India, April 2015
© 2017 IBM Corporation7
[India] Ganga – Local Ground Situation @ Varanasi (Assi/ Tulsi Ghats) + Patna
Photos of/ at Assi/ Tulsi Ghat, Varanasi on 25 March 2015 during 1700-1800 Hrs
Assi Ghat post recent cleanup Bathing on Tulsi Ghat
A nullah draining into GangaA manual powered boat
Photos at Gandhi Ghat, Patna on 18 March 2015during 1700-1800 Hrs
Common scene around Indian water bodies
© 2017 IBM Corporation8
[India] Ganga – Local Ground Situation Near Haridwar
Photos at different spots along Ganga at Haridwar-Rishikesh’s 70Km stretch during Feb-Apr 2016
© 2017 IBM Corporation9
Decision Example –River Water Pollution
§ Value – To individuals, businesses, government institutions– Individual Use Example – Can I take a bath? Will it cause me dysentery? Can I drink this water? Can I use this water for irrigating my kitchen garden? Can I eat the fish I catch?
– Business Use Examples – How should govt spend money on sewage treatment for maximum disease reduction? If an industry is being set-up, which site is best among feasible alternatives with least impact and latest technology?
§ Data – Quantitative as well as qualitative– Dissolved oxygen, – pH, – … 30+ measurable quantities of interest
§ Access –– Today, little, and that too in water technical jargon– In pdf documents, website
Key Idea: Can we make insights available when needed and help people make better decisions?
© 2017 IBM Corporation10
GangaWatch Video Demo
Video at: https://youtu.be/MbVvVGsZoTo
GangaWatch App on Android store, https://play.google.com/store/apps/details?id=com.ibm.research.gangawatch
GangaWatch blog - https://www.linkedin.com/pulse/ganga-watch-app-using-water-data-every-day-decisions-srivastava
© 2017 IBM Corporation11
Initiative: UK
§ Summary– A web-based interface specific to bathing atmajor water-bodies
§ Reference– https://environment.data.gov.uk/bwq/profiles/
© 2017 IBM Corporation12
Initiative: New South Wales, Australia
§ Summary– Web and Mobile based tools to visualize water availability in river bodies in New South Wales
§ Reference– http://waterinfo.nsw.gov.au/– http://realtimedata.water.nsw.gov.au/water.stm
© 2017 IBM Corporation13
Initiative: US Geological Survey§ Summary– Collects water data from different locations, makes available online along with web services to access them programmatically
– Interpretation of goodness for a purpose left to user;; Clear Water Act and state laws impact interpretation
§ Reference– https://www2.usgs.gov/water/, https://waterwatch.usgs.gov/wqwatch/ https://www.epa.gov/laws-regulations/summary-clean-water-act
© 2017 IBM Corporation14
Initiative: Jefferson Project at Lake George
§ Summary– Joint collaboration between academia and industry, project tries to collect large-scale water data and do hydrological modeling;; help promote conservation
– Example of outcome: Zooplankton Rapidly Evolve Tolerance to Road Salt, https://www.rdmag.com/news/2017/01/zooplankton-rapidly-evolve-tolerance-road-salt
§ Reference– https://fundforlakegeorge.org/JeffersonProject
© 2017 IBM Corporation15
Initiative: RiverKeeper for Hudson River
§ Summary– Impact on recreation activities due to human waste contamination
§ Reference– http://www.riverkeeper.org/water-quality/hudson-river/
© 2017 IBM Corporation16
Initiative: Sewage Processing in Spain
§ Summary– Optimizing operation of sewage processing regularly rather than statically / seasonally.
– In city of 2 million people in Spain, an operation optimization system showed a dramatic 13.5 percent general reduction in the plant’s electricity consumption, a 14 percent reduction in the amount of chemicals needed to remove phosphorus from the water, and a 17 percent reduction in sludge production.
§ References– https://www.ibm.com/blogs/research/2016/08/used-iot-data-optimize-wastewater-treatment/
– Operational optimization of wastewater treatment plants: a CMDP based decomposition approach, Zadorojniy, A., Shwartz, A., Wasserkrug, S. et al. Ann Oper Res (2016). At: http://link.springer.com/article/10.1007/s10479-016-2146-z
© 2017 IBM Corporation17
Analytics: Potential Use CasesS.No.
Stakeholder
Use case Data Analyticaltechniques
1 IT Identifying and removing outliers, data validation
Sensor data Data mining (outlierdetection)
2 Individual Which bathing site to use? Sensor data, ghat data Rule-based decision support
3 Individual/ Economy
What crops can I grow that will flourish in available water?
Sensor data, crop data Distributed data integration, co-relation
4 Institution Determine trends/anomalies in pollution levels
Sensor data, weather data
Time series analysis,anomaly detection
5 Institution Attribute source of pollution at a location
Sensor data, demographics, industry data
Physical modeling, inversion
6 Institution Sewage treatment strategy and operational planning
Sensor data, demographics, STP data
Multi-objectiveoptimization
7 Institution Promoting wildlife/ dolphins Sensor data, wildlife data Rule-based decision support
8 Institution Using waterways for commercial shipping
Sensor data, commercial shipping routes
Optimization, planning
17
© 2017 IBM Corporation18
Water Basics and Problem
© 2017 IBM Corporation19
Water Cycle (aka Hydrological Cycle)
Source: Economist, May 20, 2010
© 2017 IBM Corporation20
Fresh Water: Supply and Demand
Source: Economist, May 20, 2010
Supply Demand
© 2017 IBM Corporation21
Water Usage Trivia
© 2017 IBM Corporation22
Fresh Water Stress: Spatial Distribution
Freshwater stress will affect both developed and developing nations. Billions will be affected.
© 2017 IBM Corporation23
Water Challenges Summary
§ Increasing demand due to– Population– Changing water-intensive lifestyle– Industrial growth
§ Shrinking supplies– Erratic rains due to climate change– Sewage / effluent increase
§ Poor management– Below cost, unsustainable, pricing– Delayed or neglected maintenance
Water is the next flash point for wars
© 2017 IBM Corporation24
Better Information Flow is Critical for Better Water Flow
“One barrier to better management of water resources is simply lack of data — where the water is, where it's going, how much is being used and for what purposes, how much might be saved by doing things differently. In this way, the water problem is largely an information problem. The information we can assemble has a huge bearing on how we cope with a world at peak water.”Source:Wired Magazine, “Peak Water: Aquifers and Rivers Are Running Dry. How Three Regions Are Coping”, Matthew Power, April 21st, 2008
The nature of water management must rapidly evolve
From To
Manual Data Collection Automated SensingManaging Collaboratively
Intermittent Measurement Real-Time MeasurementMultiple Data Sets Data Integration“Guesstimation” Tools Modeled Decision SupportCommodity Pricing Value PricingTactical Problem Solving Strategic Risk Management
Managing in Isolation
© 2017 IBM Corporation25
Decisions Within Water Life Cycle
11
1
Water Sources• How much water is available?• What is the quality of the water?• How secure is the water?• How is the water changing over time?• How is/should the water be mixed?• Do we comply with regulations/water rights?
1
2
2 2
Modeling and forecasts•Feeds to modeling (e.g. SWIM)
• Meteorological models• Hydrological Models
• Groundwater protection• Feeds to storm water management• Alerts to municipalities/first responders
22
2 2
2
2
33
33
3 3
3
3
3
Metering Analytics• How is the water being used?• When is the water being used?• When patterns change dramatically, what to do?• When the sum of the parts is less than the whole.. Where is the water going?
3Combined Sewer Overflow• Was this truly an event?• How large was it?• What was the composition?• Can the existing capacity be used better?• How do I notify downstream entities?• How do I coordinate upstream agencies?• How can we backtrack to owners?
4
4
4
4
4
4
4
4
Environmental Analytics• How is the environment changing?• How do we notify interested parties as it changes?• How does the entire system react to changes?• How can we predict changes?
5
55
5
5
5
5
5
5
Advanced Water Management Capabilities
© 2017 IBM Corporation26
Towards Solution: Data & Infrastructure
© 2017 IBM Corporation27
Water Pollution Sensing
§ Method 1: Sample collection and lab-testing– Accurate when done well;; rich historical data which is under-utilized;; open data technologies can make them accessible
– Time-consuming, costly and thus feasible for a few places only at a time– Only quantitative– Science: lab tests, sample collection
§ Method 2: Real-time sensing– Timely, inexpensive– Some important parameters are NOT feasible but can be inferred (e.g., BOD, FC)– Only quantitative– Science: how to deploy sensors and analyze data
§ Method 3: Crowd-sourcing– Timely, inexpensive– Only qualitative assessment– Practical for India with people and mobiles– Science: Combining qualitative and quantitative data
© 2017 IBM Corporation28
Quantitative Sensing ScopeDimension Yamuna | Hindon| GangaScenario focus General, Agriculture
Real-time measurement DO, pH, conductivity, turbidity
Lab / samples BOD, COD, FCC
Sensing COTS sensors, Machine learning, In-lab test
Data ingestion Bluemix cloud, Cloudantdatabase
Primary§ Temp§ ORP§ D.O§ EC§ Turbidity§ Pressure§ Nitrate § GPS Lat§ GPS Long
Secondary
• Resistivity• TDS• Salinity• SeaWater Sigma
Sensor Measures
§ Microcontroller§ Sensor probes§ Communicator (Shields)
§ Data Storage (Server)
© 2017 IBM Corporation29
Water Qualitative Data Via Crowdsourcing –NeerBandhu App
Data at http://nalanda.haifa.il.ibm.com/naturetrack/visualization.php
© 2017 IBM Corporation30
Creek Watch – Crowd Sourced Water Information Collection
As on 14 Oct 2014
© 2017 IBM Corporation31
Location: http://creekwatch.researchlabs.ibm.com/call_table.php
~3120 data points in 4 years from around the world
As on 14 Oct 2014
© 2017 IBM Corporation32
Gaps Filled by Our Approach
§ High spatial and temporal resolution (real-time)– Current data are at low resolution of few places and limited time points;; limits usage in applications
– Use floating platform and real-time sensor to collect GPS-enabled data
– Use location to re-create water body condition
§ New source of data (qualitative;; crowd-sourcing) § Fusion of historic and new real-time data on single platform with safety levels and purpose
§ Future: contextualize quantitative data with qualitative inputs for data validation and stakeholders buy-in
© 2017 IBM Corporation33
Demo: GangaWatch (1/2)
Data Covering Ganga BasinFine-grained Geo-tagged Data from a Real TimeRun on Yamuna
© 2017 IBM Corporation34
Demo: GangaWatch (2/2)
© 2017 IBM Corporation35
Blue Water Architecture: Water Data
Data (Multiple Sources)
• Stationary Stations• Labs• Mobile Stations• Historical Data• ….
Applications
Web Service
Authenticated:• Data Upload
(Excel, CSV)• Meta Data
Upload• Auto Meta Data
generation
NoSQL Database: Cloudant
Stores: Data (Semi-Structured Data)Meta Data (Places, Limits)
IBM Cloud : Bluemix
Data Query Support
Spatio-temporal QueriesMeta Data Based Queries
https://bluewater.mybluemix.net/
Rest API
AuthenticatedMultiple queries are supported
1 2
3
4
56
© 2017 IBM Corporation36
BlueWater In-Depth: Web Service
Authenticated Data Uploadhttps://bluewater.mybluemix.net
Manual Places Definitions
Limits on Water Quality
Water Quality Data of Parameters in CSV
Water Quality Data of Parameters in Excel
© 2017 IBM Corporation37
BlueWater In-Depth: REST API
License Key Generation:https://bluewater.mybluemix.net/license.jsp
Add few basic details and get license online.
Supported queries§Spatial query§Temporal query§Mixed query§Meta data query
© 2017 IBM Corporation38
BlueWater In-Depth: REST API
API URL: https://bluewater.mybluemix.net/query (All results are returned in json)Returns the info about data: Parameters, Units, statistics of data
Spatial queryInputs: Latitude, Longitude, Range and LicenseReturns: Data in Json
Example:https://bluewater.mybluemix.net/query?Latitude=28.66501&Longitude=77.2393&Range=10&Lic=XXX
Data Returned:
"source":"live_sensor_data","lat":28.66505,"lng":77.23923,"unix_timestamp_creation":1485297242,"userid":"ibmadmin","date":"2015/12/16","time":"12:02:48","datetime":1450267368,"Water_Temperature":15.94,"pH":7.82, ……
(Lat,Lng)
R
Haversine
© 2017 IBM Corporation39
BlueWater In-Depth: REST API
Temporal queryInputs: Date_1, Date_2 and License
Time_1, Time_2 and License
Returns: Data in JsonExample:https://bluewater.mybluemix.net/query?Date_1=2015/12/16&Date_2=2015/12/18&Lic=XXXhttps://bluewater.mybluemix.net/query?Time_1=1450267414&Time_2=1450267426&Lic=XXX
Mixed queriesExamples:
https://bluewater.mybluemix.net/query?Latitude=28.66501&Longitude=77.2393&Range=1000&Time_1=1450267414&Time_2=1450267426&Lic=XXXhttps://bluewater.mybluemix.net/query?Latitude=28.66501&Longitude=77.2393&Range=10&Date_1=2015/12/16&Date_2=2015/12/16&Lic=XXX
Date = YYYY/MM/DD Time = Unix Timestamp(UTC)
© 2017 IBM Corporation40
BlueWater In-Depth: REST API
Meta Data queryInputs: Places parameters and License
Limits and License
Returns: Data in Json Example:https://gw-ser1.mybluemix.net/query?Places-By-User=UserID&Place-Type=type&Lic=XXXhttps://gw-ser1.mybluemix.net/query?Limits-By-User=UserID&Limit-Type=drinking&Lic=XXX
Rest API Demo (Demo Usage Code in Java)https://github.com/sandeep-iitr/BlueWater_REST_API_DEMO
© 2017 IBM Corporation41
BlueWater In-Depth: GangaWatch Using REST API
Rest API
AuthenticatedMultiple queries are supported
© 2017 IBM Corporation42
Experience Working in the Field
Water-bodies of focus: (1) Hindon, sub-tributary Yamuna, tributary of Ganga(2) Yamuna, tributary of Ganga(3) Ganga
© 2017 IBM Corporation43
Experience Working in the Field - Hindon
River: Hindon (sub-tributary Yamuna, tributary of Ganga)
© 2017 IBM Corporation44
Hindon, Near Meerut, India (Sep 2015)
Places on HindonRiver
1. Kinoni Village (K)2. Barnawa (Ba)3. Budhana Road (Bu)4. Budhana (Bu-City)5. Titavi, Muzaffarnagar (T)6. Saharanpur
1
2
3
4
5
© 2017 IBM Corporation45
Hindonon the Ground
Kinoni Village (K)
Barnawa (Ba)
1
2
3
4
5
© 2017 IBM Corporation46
Data Around Hindon
46
Historical Qualitative Collected
© 2017 IBM Corporation47
Location Findings
K Ba Bu Bu-City TDepth 3 feet 1-2 feet 5-6 feet 1 feet 3 feetBoat navigation
100 m 100 m 200 m 10-20 m 50 m
Water flow Medium Low Medium Low FastVisible Trash
Low Low Medium High Low
Distance to major highway
Far Near Near Near Near
© 2017 IBM Corporation48
Insights from Hindon
§ Boat cannot go at many places– Paddle boat may be more useful than powered boat– At one stretch, boat can go for 100m max at most places visited
§ Mobile data collection was done and useful with NeerBandhu;; GPRS signal strong during the whole trip
§ Diseases prevalent– Humans: Cancer, gastro, infertility, skin diseases– Animals: infertility, sudden death
• Current water usage: Bathing cattle, irrigating fields, drinking by buffalo§ Impact of sensed data/ use cases: using river water data for– Plantation of trees along the river bed– Distribution of water filtration systems or setup of overhead tanks by government in villages (Note: current data may itself be useful)
– Spreading awareness about river water usage for vegetables has lead to change
© 2017 IBM Corporation49
River: Yamuna (tributary of Ganga)
Experience Working in the Field - Yamuna
Reference:
A multi-sensor process for in-situ monitoring of water pollution in rivers or lakes for high-resolution quantitative and qualitative water quality dataSukanya Randhawa, Sandeep S Sandha and Biplav Srivastava, 14th IEEE/IFIP International Conference on Embedded and Ubiquitous Computing (EUC 2016), August 2016.
© 2017 IBM Corporation50
Real-Time Sensor Deployment
© 2017 IBM Corporation51
Day 1 -‐ multiple anchoring approaches for real-‐time sensor on another day (16 Dec) in 2-‐3 km stretch
16-‐Dec-‐15
Location Name DescriptionSample -‐collected Sample -‐ testing Sensor @site
Realtime (Stretch) Neer Bandhu
1Point 1 [A] Nigambodh, in waterY
Y (ph, DO, Temp, Turb, Cond, BOD, FCC) Y Y
2Point 2 [B] Y Y3Point 3 [C] ITO bridge Y Y Y4Point 4 [D] Y Y
5Pointe 5 [E] YY (ph, DO, Temp, Turb, Cond) Y
6Point 6 Moving (7-‐8 Kmph) Y7Point 7 Moving (10 Kmph) Y Y
8Point 8 Drain YY (ph, DO, Temp, Turb, Cond) Y Y
9Point 9 With Ted buoy Y
© 2017 IBM Corporation52
Dec 16Example Run
16/12/15 13:59:34
16/12/15 13:46:50
• ~12 minute downstream travel
• 765 data points
© 2017 IBM Corporation53
Turbidity in Yamuna – measured on 16th Dec, 2015Data min: 56.7Data max: 127Gradient: Default
56
91
127TurbidityFNU
© 2017 IBM Corporation54
Day 2 -‐ Covered ~7-‐8 km one-‐way on one of the days(18 Dec) roughly covering 33 % of the navigable stretch of Yamuna in Delhi (22 km one-‐way).
18-‐Dec-‐15
Location Name DescriptionSample -‐collected Sample -‐ testing Sensor @site
Realtime (Stretch) Neer Bandhu
1Point 21 [AA] Nigambodh, in waterY
Y (ph, DO, Temp, Turb, Cond, BOD, FCC) Y Y
2Point 22 [AB] Past rope (ISBT) Y Y3Point 23 [AC] 2nd rope Y Y
4Point 24 [AD] Drain YY (ph, DO, Temp, Turb, Cond) Y Y Y
5Pointe 25 [AE] Drain Y Y6Point26 [AF] Drain, gurudwara Y Y
7Point 27 [AG] Wazirabad bridge YY (ph, DO, Temp, Turb, Cond) Y Y Y
8Point 28 [AH] Majnu ka tila, greenery Y Y
9Point 29 [AI] 1st rope, ISBT Y
© 2017 IBM Corporation55
Dec 18Example Run
2015/12/18,12:13:45
2015/12/18,12:51:37
• ~38 minute upstream travel
• 2273 data points
© 2017 IBM Corporation56
Turbidity in Yamuna – measured on 18th Dec, 2015Data min: 50Data max: 144.4Gradient: Default
50
97
144TurbidityFNU
56
91
127TurbidityFNU
Reference:Turbidity of 16-Dec
© 2017 IBM Corporation57
Turbidity and Conductivity in Yamuna – measured on 18th Dec, 2015
50
97
144TurbidityFNU
1268
1424
1582ConductivityµS cm
© 2017 IBM Corporation58
Lab Samples and Traditional Testing
© 2017 IBM Corporation59
16/12/2016 18/12/2016Temp(°C) 15.93 15.34pH 7.82 7.81ORP(mV) -182 -86.4D.O(mg/L) 3.76 3.53EC (µS/cm) 1604 1279Turbidity (F.N.U) 84.25 66.9BOD (mg/L) 46 28.2Fecal Coliform(No./100 mL) 430 210
Change in parameters measured for two different days
Sensor
Lab
© 2017 IBM Corporation60
NB Qualitative Data
http://nalanda.haifa.il.ibm.com/naturetrack/visualization.php
© 2017 IBM Corporation61
Correlating RT Sensor and Crowd Data to GetVerifiable Data!
© 2017 IBM Corporation62
Towards Solution: Usage Scenario
© 2017 IBM Corporation63
Towards Solution:
Usage Scenario – Inspection
Reference:
Protecting the NECTAR of the Ganga River Through Game-Theoretic Factory Inspections,Benjamin Ford, Matthew Brown, Amulya Yadav, Amandeep Singh, AruneshSinha, BiplavSrivastava, Christopher Kiekintveld, Milind Tambe,14th International Conference on Practical Applications of Agents and Multi-Agent Systems, Sevilla, Spain, June 1-3, 2016
© 2017 IBM Corporation64
Art of Possible
Tannery Example: Kanpur, India
© 2017 IBM Corporation65
Background of Leather Tanning Problem
• > 700 tanneries in Kanpur– Employing > 100,000 people– Bringing > USD 1B revenue
• Discharge water after leather processing to river or Sewage treatment plants (STPs)– Requirement
• Must have their own treatment facility• Or, have at least chrome recovery unit
– But don’t implement due to costs which is a burden to main operations• Installation• Operations : electricity, manpower, technology upgrade, …
– State pollution board is supposed to do inspections to enforce but doesn’t perform effectively
• Government’s STPs do not process chrome, the main pollutant • Knee-jerk reaction: 98 tanneries banned in Feb 2016 by National Green Tribunal;; more threatened
© 2017 IBM Corporation66
Flow Chart of Tanning
Source: home.iitk.ac.in/~sgupta/tannery_report.pdf
© 2017 IBM Corporation67
Water Pollutant Standards for Tanning
Pollutant standards to maintain depends on whether effluent discharged directly to river or to STP in-drain
•To river directly– Chrome: < 1 mg/l– Sulphide: < 2 mg/l– Suspended solids: < 100 mg/l– Ph: 6.5 – 9– COD: < 250 mg/l
•To STP (after which, STP will process) – Suspended solids: < 600 mg/l– Ph: 6.5 – 9
• Current state– Chrome: 45 mg / l– Other heavy metals are also at alarming level: arsenic, mercury, nickel
© 2017 IBM Corporation68
Consequences of Contaminated Water in Kanpur
§ Poor public health in Kanpur and downstream
§ Affects poorest the most and government eventually– Rural people in India (70%) spend at least Rs.100 each year for the treatment of water/sanitation-related diseases. Which is approximately same as Central Govt’s Health budget (Rs 6700 crore;; $US 1.1B);; doesn’t factor the costs to urban India (30%)
– Similar implications likely for Kanpur area too
§ Stresses future water supply: lowering ground water table§ Slows future industrial growth and thus economy
© 2017 IBM Corporation69
NECTAR: Nirikshana for Enforcing Compliance for Toxic wastewater Abatement and Reduction.
Setting• Attackers
• M sites with N factory units each• When inspection at a site happens, all units know
• Defenders• Inspectors base office is fixed• Inspection team consists of
• Environment Inspectors• Security personnel• Transport provider / drivers
• Inspection team starts and ends at their office
• Security and transport can vary daily
• Objective• Create daily inspection plan which minimizes violation over a time period
Joint work with USC, USA
© 2017 IBM Corporation70
Demo: NECTARhttp://teamcore.usc.edu/people/benjamin/ganga/gangaDemonstration.htm
"Very promising approach. Use of decoys and data-driven random were not known in the inspection community where it was known that random could help. Surprise elements of decoys and variable fines provide new factors for compliance. The data from drone monitoring can help improve the plans significantly as future work."
Dr. Venkatraman Rajagopalan, IAS Ex-Secretary, Ministry of Environment, Forests and Climate Change, and Ex-Chairman, Central Pollution Control Board, India
© 2017 IBM Corporation71
AI in the Scenario
§ Inspection generation a green security game - StackelbergSecurity Game
§ Solve as MDP to generate inspection plan and schedule§ Explain plan and what-if questions using simulation and rules
§ Advanced– Utility-driven planning - Use pollution data to update inspection objective and replan
– Mechanism design - Determine optimal fine strategy
© 2017 IBM Corporation72
Towards Solution:
Usage Scenario – Tourism Impact
Reference:
An Open, Multi-Sensor, Dataset of Water Pollution of Ganga Basin and its Application to Understand Impact of Large Religious Gathering,B Srivastava, S Sandha, V Raychoudhury, S Randhawa, V Kapoor, A AgrawalarXiv preprint arXiv:1612.05626
© 2017 IBM Corporation73
Use-Case: Understand Impact of a Large-Scale Religious cum Tourism Event
§ Haridwar Ardh Khumbh Mela 2016– January 1, 2016 to April 30, 2016– Over 100 millions attended;; Many took dip in river at select spots
– Major bath sub-events during the period have high burst of visitors
§ Question– How much does human activity impact river?– Where is the impact highest? Of what kind?
Sources:1. https://en.wikipedia.org/wiki/Kumbh_Mela2. http://www.kumbhamela.net/kumbha-mela-haridwar.html2. http://www.thegreatananda.com/ardh-kumbh-mela-2016-haridwar/
Date (2016) Day Main Bathing Event (Snan)
14th January Thursday Makar Sankranti12th February Friday Vasant Panchami22nd February Monday Magh Purnima7th March Monday Mahashivratri7th April Thursday Chaitra Amavasya
8th April Friday Chaitra Shukla Pratipada
14th April Thursday Mesha Sankranti15th April Friday Ram Navami
22nd April Friday Chaitra shuklaPurnima
© 2017 IBM Corporation74
Ardh Kumbh 2016, Haridwar
Territorial Bird’s Eye View: ~76 KM (Road Distance)
© 2017 IBM Corporation75
Turbidity Variations Feb 27-28, 2016
Turbidity values at different places (places marked red have turbidity value above the drinking range, places marked blues ha turbidity value in range of drinking water)
© 2017 IBM Corporation76
Data Collection Points around Har-ki-pauri, HaridwarFeb 27-28, 2016
Carrying sensor on a buoy for long stretch was not possible due to water speed.
45+ places from Rishikesh to Ganga Canal (Roorkee) (75+ KM)
© 2017 IBM Corporation77
Pollution on Major Bath Day around Har-ki-pauri, HaridwarMarch 7, 2016
Turbidity values at different places (places marked red have turbidity value above the drinking range, places marked blues ha turbidity value in range of drinking water)
© 2017 IBM Corporation78
© 2017 IBM Corporation79
AI in the Scenario
§ Data cleaning, normalization, missing values§ Quantitative to qualitative data conversation
§ Water Quality Index
§ Aggregate Binary Clustering using parameters with opposite polarities (e.g., Dissolved Oxygen, Turbidity), interval functions (e.g., pH)
§ Advanced– Generating consolidated qualitative assessment across multiple parameters
– Explaining and validating assessment
© 2017 IBM Corporation80
Discussion
© 2017 IBM Corporation81
Recap: Deliver Value From Water (Pollution) Data
§ Government for business decisions– Source attribution, inspection– Sewage treatment– Public Health
§ Individuals for personal decisions– Bathing (Religious, Lifestyle)– Recreation– Community practices
© 2017 IBM Corporation82
AI Research Issues
§ Sensing– Deciding sensors to use (multi-objective optimization)– How to sense cost-effectively? (Quantitative sensing)
• Install sensors• Ensure sensor up-keep, inspections• Decide sampling rate for sensors
– How to involve people-as-sensors? (Qualitative sensing)• Use people as inspectors (increase resources for defense)• Mobilization when needed on short notice• Devising incentives for contribution
© 2017 IBM Corporation83
AI Research Issues
§ Interconnection– Within water: quantitative and qualitative estimates;; relation between fresh and sewage water
– Across domains: energy implications on water management, physical safety, waste water treatment
§ Analytics– Decision-support (optimization, planning, scheduling) for organizing large-scale human activities
– Optimizing short-term and long-term investments (capex/ opex) for maximizing overall-value from invested water assets
– Pricing to incentivize water conservation and behavioral change
© 2017 IBM Corporation84
Water Data Standards§ A dis-organized collection of regulations varying by purpose (e.g., drinking, agriculture) and regions (country, state).
§ Conflicts may occur;; some data may have privacy considerations (e.g., flow volume);; standards often mixed with testing methods
§ WHO: water quality standards (from health perspective). At: http://www.who.int/water_sanitation_health/publications/whoiwa/en/
§ US: Environment Protection Agency’s water quality standards– https://www.epa.gov/wqs-tech– http://water.epa.gov/scitech/swguidance/standards/– https://www.epa.gov/wqs-tech/state-specific-water-quality-standards-effective-under-clean-water-act-cwa
§ Europe: http://ec.europa.eu/environment/water/index_en.htm
§ India: Central Pollution Control Board, State Pollution Boards– http://www.cpcb.nic.in/Water_Quality_Criteria.php– CPCB guidelines for real-time data, http://www.cpcb.nic.in/FinalGuidelinse.pdf– CPCB's list of pollutants, http://cpcb.nic.in/list_of_parameters.pdf
© 2017 IBM Corporation85
Call for Action
§ Use water data from BlueWater and find new insights– Use APIs– Build apps and/ or reuse GangaWatch app code
§ Collect and contribute your own data– APIs exist to upload;; registration need
§ Focus on a water use-cases and look at how you can formulate a basic problem;; solve them– Fishing– Water-borne public health– …
© 2017 IBM Corporation86
Conclusion
§ We highlighted the importance of water and gave a snapshot of potential for water informatics
§ Presented the BlueWater platform and GangaWatch app§ Shared field-experience collecting and using data§ Demonstrated use-cases of providing decision-support using AI techniques in water context
§ Tutorial can serve as a resource for others to contribute
© 2017 IBM Corporation87
References / Resources
§ Peter Gleick et al, The World's Water, Volume 8, The Biennial Report on Freshwater Resources, 2014
§ IEEE Spectrum Special Report: Water vs Energy (June 2010), http://spectrum.ieee.org/static/special-report-water-vs-energy
§ Economist Special Report (May 22, 2010), For Want of a Drink: Special Report on Water
§ World Bank– InvestIng In Water Infrastructure:Capital, Operations and Maintenance, Diego J. Rodriguez, Caroline van den Berg and Amanda
McMahon, 2012, At: http://water.worldbank.org/sites/water.worldbank.org/files/publication/water-investing-water-infrastructure-capital-operations-maintenance.pdf
– High and Dry: Climate Change, Water, and the Economy, http://www.worldbank.org/en/topic/water/publication/high-and-dry-climate-change-water-and-the-economy
– World Bank India Ground Water Report, Mar 2010
§ Binayak Ray: Water – The Looming Crisis in India, 2008, ISBN-13: 978-0739126028
§ IBM– Smarter Water Management Thought Leadership , http://www.ibm.com/smarterplanet/water– Smarter Water Management Solutions Home Page, http://www.ibm.com/green/water– IBM GIO Report on Oceans and Water, http://www.ibm.com/ibm/gio/water.html– IBM Water Management Pains Summary Report, http://www-935.ibm.com/services/us/gbs/bus/pdf/ibm-water-pains-report-jan09.pdf
© 2017 IBM Corporation88
Blue Water ResourcesApps / tools§ GangaWatch app for water pollution information on Google playstore. See description here in the blog.(Jan 2016
§ NeerBandhu (Water Friend) app on Google playstore. See description here in the blog.(Nov 2015)
Data§ From Blue Water/ GangaWatch, REST APIs are at: http://researcher.watson.ibm.com/researcher/view_group_subpage.php?id=7142 .
§ From Neer Bandhu (Water Friend), also available from the mobile app. Find at: http://nalanda.haifa.il.ibm.com/naturetrack/visualization.php
§ From Creek Watch, also available from web site. Find http://creekwatch.researchlabs.ibm.com/call_table.php
Blogs§ Ganga Watch and Blue Water Revisited – Collected Data is Now Externally Available, As-is,Aug 2016.
§ Ganga Watch – An App for Using Water Data in Every Day Decisions, Jan 2016.
§ Two Days on Yamuna – Collecting Water Pollution Data is Both Simple and Complex, Dec 2015.
§ Neer Bandhu, for Water Friends, Nov 2015.§ Can Game Theory Help Clean the Ganga? Randomized Inspection is a Case in Point, Benjamin Ford, BiplavSrivastava*, Milind Tambe, Sep 2015. At data.gov.in'senvivonmentcommunity.
Papers§ An Open, Multi-Sensor, Dataset of Water Pollution of Ganga Basin and its
Application to Understand Impact of Large Religious Gathering, Biplav Srivastava, Sandeep Sandha, Vaskar Raychoudhury, SukanyaRandhawa, Viral Kapoor, Anmol AgrawalOn Arxiv at: http://arxiv.org/abs/1612.05626, December, 2016.
§ The GangaWatch Mobile App to Enable Usage of Water Data in Every Day Decisions Integrating Historical and Real-time Sensing Data,Sandeep Sandha, Biplav Srivastava, Sukanya Randhawa,Demo paper on Arxiv at: http://arxiv.org/abs/1701.08212, January 2017.
§ A multi-sensor process for in-situ monitoring of water pollution in rivers or lakes for high-resolution quantitative and qualitative water quality dataSukanya Randhawa, Sandeep S Sandha and Biplav Srivastava, 14th IEEE/IFIP International Conference on Embedded and Ubiquitous Computing (EUC 2016), August 2016.
§ The GangaWatch App and BlueWater Platform to Enable Usage of Water Data in India in Everyday Decisions Integrating Historical and Real-time Sensing Data,Sandeep S. Sandha, Sukanya Randhawa, and Biplav Srivastava,Data Flow: Grand Challenges in Water Systems Modeling, Data Management, and Integration, Louisiana State University, Baton Rouge, May 9-10, 2016.
§ Protecting the Nectar of the Ganga River through Game-Theoretic Factory Inspections, B. Ford, A. Yadav, A. Singh, M. Brown, A. Sinha, B. Srivastava, C. Kiekintveld, M. Tambe,14th International Conference on Practical Applications of Agents and Multi-Agent Systems, Sevilla, Spain, June 1-3, 2016.
§ Blue Water: A Common Platform to Put Water Quality Data in India to Productive Use by Integrating Historical and Real-time Sensing Data, Sandeep S Sandha, Sukanya Randhawa and Biplav Srivastava, IBM Research Report RI15002, 2015.