1
Prepare for Variation – Towards Using AI/Robotics Methods for Handling
Uncertainty in Agriculture Applications
Joachim Hertzberg
Osnabrück University and
DFKI Robotics Innovation Center (RIC), Osnabrück Branch
IROS Workshop on Agri-Food Robotics, Hamburg, Oct 2nd, 2015
2
1. Uncertainty in AI
2. Small Example: Sensor Data Interpretation
3. Larger Example: Combine Harvester Control
4. Wrap-up
3
Uncertainty in AI – Generic Methods
Part I Artificial Intelligence Part II Problem Solving Part III Knowledge and Reasoning Part IV Uncertain Knowledge and Reasoning
13 Quantifying Uncertainty 14 Probabilistic Reasoning 15 Probabilistic Reasoning over Time 16 Making Simple Decisions 17 Making Complex Decisions
Part V Learning Part VII Communicating, Perceiving, and Acting Part VIII Conclusions
3rd Ed., 2010
2005
… and in AI/Robotics
4
Dealing with Uncertainty – the AI Arsenal
• Probabilistic representation & reasoning – Bayesian networks – Graphical models
• Logical r&r about defaults/prototypes and preferences – Answer set programming
• R&r about time and space – Constraint-based quantitative methods, e.g., STPs – Interval-based qualitative calculi, e.g. interval algebras
Key problem for building applications: combine • representational efficiency, learnability • inferential efficiency • integration of different reasoners & their representations
5
Uncertain ≠ Chaotic!
• Literature (AI, Robotics) often mentions “unstructured environments”
• Nearly always they mean “not 150% controlled”: variety, lack of knowledge, clutter, occlusion, dynamics, … – but normally a high degree of structure! Structure is often known and/or learnable!
• R&r can exploit this structure, if part of a closed-loop control! (and so can learning, perception, action, …)
6
1. Uncertainty in AI
2. Small Example: Sensor Data Interpretation
3. Larger Example: Combine Harvester Control
4. Wrap-up
7
Own Contributions: Interpreting 3D Point Clouds
… but until now mostly in non-agriculture applications
8
Rigor in Structure – Variation in Detail
Photos: Wikipedia/english “Grape” Image: Wikipedia/deutsch “Weinrebe”
9
Model-Based Phenotyping Steinhage et al.: A Model-Based Approach to High Performance Phenotyping, 2012 Univ. Bonn, CROP.SENSe
Grammar-Based Stem-Skeleton Interpretation Relational Growth Grammar (cf. skeleton models)
Probabilistic Sampling of 3D Scan Structuring Hypotheses Monte-Carlo-based sampling in hypothesis space
10
Model-Based Phenotyping, ctd.
Result Hypotheses • Length of rachis: 8.50 cm
• Av. pedicel length: 0.95 cm
• # berry groups: 18
• # nodes on rachis: 9
• # twigs with sub-twigs: 3
• # terminal pedicel twigs: 9
If this is often enough good-enough (“satisficing”) information …
11
1. Uncertainty in AI
2. Small Example: Sensor Data Interpretation
3. Larger Example: Combine Harvester Control
4. Wrap-up
12
Cooperative Corn Harvesting Scenario
S. Scheuren, S. Stiene, R. Hartanto, J. Hertzberg, M. Reinecke. Spatio-temporally Constrained Planning for Cooperative Vehicles in a Harvesting Scenario. KI – Künstliche Intelligenz, 27(4):341-346, 2013
Harvested area Non-harvested area Combine harvester Tractor Truck
Driving Paths
Rendezvous point
13
Route Planning for the Tractor Outer field border Inner field border
Driving traces Headland trace Unloading spot
Route graph Overloading window
14
Motion Planning, Structure
Tractor Harvester
Motion Planning
Approximate nominal path
SBPL lattice planner2
Graph search
Approximate nominal path
SBPL lattice planner2
15
Motion Plan for the Tractor
Outer field border Inner field border
Driving traces Headland trace Unloading spot Path of tractor
Overloading window
16
Variety #1: Machine Parameters
Motion Planning ?�Steering system
Path
Sensor data
max.
No AI – just engineering!
17
Variety #2: Change of Yield/Zone
• Planning based on prior estimation (yield/zone in the past)
• Significant variation happens – less yield is no problem – more yield may require earlier overload
• No robust solution based on ad-hoc methods
Solution Approach
• Hierarchical hybrid (causal, temporal, spatial, resource) planning • CHIMP: Conflict-driven Hierarchical Meta-CSP Planner • S. Stock, M. Mansouri, F. Pecora, J. Hertzberg. Online Task Merging with a
Hierarchical Hybrid Task Planner for Mobile Service Robots. IROS 2015
18
How to reduce this huge hybrid search space?
Reason about diffe- rent knowledge forms in an integrated way!
HTN Task Decomposition
Meta-Constraint reasoning1
1Cesta, Oddi, Smith: “A constraint-based method for project scheduling with time windows”, 2002. �
Hybrid Search Space
19
• High-level requirements modelled as meta-constraints on top of a ground-CSP1�
• Hybrid planning as enforcing consistency in a family of constraint networks2�
• Meta-CSP framework3 allows to implement solvers for problems that can be cast as a meta-CSP�
1Cesta, Oddi, Smith: “A constraint-based method for project scheduling with time windows”, 2002. �2Mansouri, Pecora: “More knowledge on the table: Planning with space, time and resources for robots”, 2014 �3http://metacsp.org�
Meta-Constraint Reasoning
20
• Integrates HTN-Planning into meta-CSP reasoning �• Represents initial state and resulting plan in constraint networks�• Creates partially-ordered plans �• Plans may contain actions that can be executed in parallel�• Allows online task merging of new goals into an existing plan
during plan execution �• Based on the meta-CSP Framework1�
Freely available at http://sebastianstock.github.io/chimp
1http://metacsp.org
CHIMP
21
… and How Can CHIMP Go Corn Harvesting?
• Model constraints in hybrid search space – temporal (e.g., durations, duration variations), – spatial (e.g., tracks), – resource (e.g., capacity, inflow)
• Model plan change strategies as meta-constraints
• Apply plan change strategies at execution time if needed; keep to domain representation
Work in progress!
22
1. Uncertainty in AI
2. Small Example: Sensor Data Interpretation
3. Larger Example: Combine Harvester Control
4. Wrap-up
23
Wrap-Up
• AI has an arsenal of established generic methods for dealing with variations
• Extending them is subject of intensive current research
• Their integration may be challenging if more than one type of representation is needed
• Yet what are the alternatives? – Ad-hoc methods lack understanding/rigor – Specific methods are hard to integrate
• May be a win-win for agricultural applications and AI!
24
CHIMP paper: cf. IROS 2015 proceedings CHIMP planner: http://sebastianstock.github.io/chimp/ �
Meta-CSP framework: http://metacsp.org
Thanks to … … my Staff • Kai Lingemann • Stephan Scheuren • Stefan Stiene • Sebastian Stock • Astrid Ullrich • Thomas Wiemann
… the marion Project (2010–13) • Ronny Hartanto • CLAAS and staff • PT DLR • BMWi
… the SOILAssist Project (2015–18) • Thünen Institut • PT Jülich • BMBF
… the RACE Project (2011–14) • Masumeh Mansouri • Federico Pecora • Örebro U • Hamburg U • EU FP7