8
Short-Term Human-Robot Interaction through Conditional Planning and Execution Valerio Sanelli 1 , Michael Cashmore 2 , Daniele Magazzeni 2 , Luca Iocchi 1 1 Department of Computer, Control, and Management Engineering Sapienza University of Rome, Italy 2 Department of Informatics, King’s College London, UK Abstract The deployment of robots in public environments is gaining more and more attention and interest both for the research op- portunities and for the possibility of developing commercial applications over it. In these scenarios, proper definitions and implementations of human-robot interactions are crucial and the specific characteristics of the environment (in particular, the presence of non-expert untrained users) makes the task of defining and implementing effective interactions particularly challenging. In this paper, we describe a method and fully implemented robotic system using conditional planning for generating and executing short-term interactions by a robot deployed in a public environment. To this end, the proposed method inte- grates and extends two components already successfully used for planning in robotics: ROSPlan and Petri Net Plans. The contributions of this paper can be summarized as fol- lows: 1) a definition of the problem of generating short-term interaction as a conditional planning problem, 2) solution of the conditional planning problem through the integration of Contingent-FF in ROSPlan, 3) integration between ROSPlan and the Petri Net Plans execution framework, 4) full imple- mentation and testing on a real robot in several different sce- narios and open-source release of the code. 1 Introduction Short-term human-robot interactions (HRIs) are useful in many application scenarios where a robot is placed in a pub- lic space interacting with non-expert untrained users. Short-term HRI is characterized by the following aspects: (i) users are not trained and are not aware about the capabil- ities of the robot, (ii) each interaction is assumed to be per- formed with a different non-experienced user (i.e., s/he did not participate in previous interactions of the same kind), (iii) interactions are short in time. Under these conditions, the robot does not need to memorize past interactions or to adapt the current interaction based on past ones. Moreover, user profiling is also not needed. Although the interactions are short, it is typically nec- essary or convenient to generate many of them in order to realize an effective application. Thus, the problem of gen- erating many interactions is important and the use of au- tomated planning procedures, instead of manual definition, Copyright c 2017, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. brings several advantages in compactness of the representa- tion, less effort for the designer of the system, and in general for the success of overall task. Since the interactions with non-expert untrained users cannot be predicted a priori and evaluating the complete cur- rent state is not easy for the robot, classical planning is not adequate for this process. Replanning is not sufficient for this case, as without offline reasoning about different contin- gencies, it is possible for the new plan to result in a dead-end state or repeated interaction (Little and Thibaux 2007). In contrast, conditional planning has the advantage that it does not require the knowledge about a complete initial state and on-line knowledge acquisition can be performed (Hoff- mann and Brafman 2005a) without requiring the computa- tion of a complete state (Bonet and Geffner 2000). Consequently, conditional planning is a useful technique for automatic generation of multiple short-term interactions performed by robots deployed in public space. In this paper, we present a complete system where condi- tional planning and human-robot interaction techniques are integrated in a fully working robotic platform that has been deployed in many public environments. The contributions of this paper can be summarized as fol- lows: 1) a definition of the problem of generating short-term interaction as a conditional planning problem, 2) a solution of the conditional planning problem though the integration with the conditional planner Contingent-FF (Hoffmann and Brafman 2005b) in ROSPlan (Cashmore et al. 2015), 3) the integration between ROSPlan and the Petri Net Plans execu- tion framework, 4) full implementation and testing on a real robot in several different scenarios and open-source release of the code. The paper is structured as follows: we start with an discus- sion of related work in Section 2, followed by an overview of the system, in which the contributions are described. We then present the integration with the real robot and experi- ments in Section 4. Section 5 concludes the paper. 2 Related Work Planning techniques have been used in robotic applications characterized by some form of human-robot interaction. A first group of works focuses on interactions between humans and robots and on an explicit representation of hu- man activities in the planning formalism. For example, An-

Short-Term Human-Robot Interaction through Conditional ...iocchi/Teaching/HRI/papers/ICAPS_2017_pape… · Short-Term Human-Robot Interaction through Conditional Planning and Execution

  • Upload
    others

  • View
    3

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Short-Term Human-Robot Interaction through Conditional ...iocchi/Teaching/HRI/papers/ICAPS_2017_pape… · Short-Term Human-Robot Interaction through Conditional Planning and Execution

Short-Term Human-Robot Interaction through Conditional Planning andExecution

Valerio Sanelli1, Michael Cashmore2, Daniele Magazzeni2, Luca Iocchi11 Department of Computer, Control, and Management Engineering

Sapienza University of Rome, Italy2 Department of Informatics, King’s College London, UK

AbstractThe deployment of robots in public environments is gainingmore and more attention and interest both for the research op-portunities and for the possibility of developing commercialapplications over it. In these scenarios, proper definitions andimplementations of human-robot interactions are crucial andthe specific characteristics of the environment (in particular,the presence of non-expert untrained users) makes the task ofdefining and implementing effective interactions particularlychallenging.In this paper, we describe a method and fully implementedrobotic system using conditional planning for generating andexecuting short-term interactions by a robot deployed in apublic environment. To this end, the proposed method inte-grates and extends two components already successfully usedfor planning in robotics: ROSPlan and Petri Net Plans.The contributions of this paper can be summarized as fol-lows: 1) a definition of the problem of generating short-terminteraction as a conditional planning problem, 2) solution ofthe conditional planning problem through the integration ofContingent-FF in ROSPlan, 3) integration between ROSPlanand the Petri Net Plans execution framework, 4) full imple-mentation and testing on a real robot in several different sce-narios and open-source release of the code.

1 IntroductionShort-term human-robot interactions (HRIs) are useful inmany application scenarios where a robot is placed in a pub-lic space interacting with non-expert untrained users.

Short-term HRI is characterized by the following aspects:(i) users are not trained and are not aware about the capabil-ities of the robot, (ii) each interaction is assumed to be per-formed with a different non-experienced user (i.e., s/he didnot participate in previous interactions of the same kind),(iii) interactions are short in time. Under these conditions,the robot does not need to memorize past interactions or toadapt the current interaction based on past ones. Moreover,user profiling is also not needed.

Although the interactions are short, it is typically nec-essary or convenient to generate many of them in order torealize an effective application. Thus, the problem of gen-erating many interactions is important and the use of au-tomated planning procedures, instead of manual definition,

Copyright c© 2017, Association for the Advancement of ArtificialIntelligence (www.aaai.org). All rights reserved.

brings several advantages in compactness of the representa-tion, less effort for the designer of the system, and in generalfor the success of overall task.

Since the interactions with non-expert untrained userscannot be predicted a priori and evaluating the complete cur-rent state is not easy for the robot, classical planning is notadequate for this process. Replanning is not sufficient forthis case, as without offline reasoning about different contin-gencies, it is possible for the new plan to result in a dead-endstate or repeated interaction (Little and Thibaux 2007).

In contrast, conditional planning has the advantage that itdoes not require the knowledge about a complete initial stateand on-line knowledge acquisition can be performed (Hoff-mann and Brafman 2005a) without requiring the computa-tion of a complete state (Bonet and Geffner 2000).

Consequently, conditional planning is a useful techniquefor automatic generation of multiple short-term interactionsperformed by robots deployed in public space.

In this paper, we present a complete system where condi-tional planning and human-robot interaction techniques areintegrated in a fully working robotic platform that has beendeployed in many public environments.

The contributions of this paper can be summarized as fol-lows: 1) a definition of the problem of generating short-terminteraction as a conditional planning problem, 2) a solutionof the conditional planning problem though the integrationwith the conditional planner Contingent-FF (Hoffmann andBrafman 2005b) in ROSPlan (Cashmore et al. 2015), 3) theintegration between ROSPlan and the Petri Net Plans execu-tion framework, 4) full implementation and testing on a realrobot in several different scenarios and open-source releaseof the code.

The paper is structured as follows: we start with an discus-sion of related work in Section 2, followed by an overviewof the system, in which the contributions are described. Wethen present the integration with the real robot and experi-ments in Section 4. Section 5 concludes the paper.

2 Related WorkPlanning techniques have been used in robotic applicationscharacterized by some form of human-robot interaction.

A first group of works focuses on interactions betweenhumans and robots and on an explicit representation of hu-man activities in the planning formalism. For example, An-

Page 2: Short-Term Human-Robot Interaction through Conditional ...iocchi/Teaching/HRI/papers/ICAPS_2017_pape… · Short-Term Human-Robot Interaction through Conditional Planning and Execution

swer Set programming (ASP) has been used to model anefficient planner for collaborative house keeping robots (Er-dem, Aker, and Patoglu 2012). This system automaticallyextracts common sense information from the ConceptNetdataset and exploits it to better perform the given task. Socialnorms have also been considered in the planning process.For example, Human Aware Planning (Tomic, Pecora, andSaffiotti 2014) is a system based on Constraint Based Plan-ning (CBP) that allows modeling and planning with socialnorms, represented as constraints of the problem. Althoughin these works, human actions are explicitly modeled, theplanning process focuses on generating interactions as inter-leaved actions between human and robots and no actions toacquire information at execution time are considered.

A second group of works refers to human-robot collabo-ration and focuses on tasks in which human and robots per-form joint actions in the environment. Planning techniquesfrom human-robot collaboration have been implemented inthe Hierarchical Agent-based Task Planner (HATP) sys-tem (Lallement, De Silva, and Alami 2014) that enablesthe definition of collaborative tasks. Recent developments ofthis system (Fiore, Clodic, and Alami 2016) allow also theuser to interact with the robot to define the plan before its ex-ecution and the robot to infer the users’ intentions. Anotherapproach for generating plans for human-robot collaborationis described in (Milliez et al. 2015). In this work, users arecharacterized by the level of knowledge they have about thetask and the robot acts according to it, by providing more orless information during plan execution. All these methodsdo not consider the possibility of having agreements aboutthe collaborative actions at execution time. Therefore, oncethe plan is started, it is not possible to change its executionflow (unless with a replanning procedure).

In general, no conditional plans are used in the abovementioned papers and thus the on-line acquisition of in-formation during human-robot interactions is not explicitlymodeled and solved as a planning problem.

Approaches based on probabilistic models (such asMarkov Decision Processes (MDP) or Partially ObservableMarkov Decision Processes (POMDP)) can represent un-certainty in the domain and compute a policy that takesinto account such uncertainty. However, in some cases, theexecution mechanism still requires a complete evaluationof the current state and the system is not robust to errorsin this computation. In order to overcome this difficulty,some recent approaches proposed the use of different lev-els of planning. For example, integration of classical plan-ning with Partially Observable Markov Decision Processes(POMDP) is described in (Hanheide and others 2015). How-ever, continuous switching planning can still be impracticalin a dynamic world with frequent human-robot interactions.In (Iocchi et al. 2016) a particular translation of the poli-cies generated by an MDP solver have been integrated witha PNP executor to reduce the need of evaluating a completestate.

However, the main problem with methods based on prob-abilistic models is that the solutions are sensitive to the manyvalues of probabilities in the model that are often difficult todetermine.

Conditional Planning techniques have been explored inHRI in the domain of a robotic bartender JAMES (Petrickand Foster 2013). In this application the HRI is modeled asa variant of the PDDL language (Mcdermott et al. 1998) in-cluding sensing actions. Plans for the interaction are gen-erated using PKS (Planning with Knowledge and Sensing)(Petrick and Bacchus 2004). More recently, the planning ap-proach for JAMES has been extended to focus on commoninteraction patterns, such as clarification questions (Petrickand Foster 2016).

We build on these techniques, aiming to provide modelsthat are domain independent. That is, a more general modelthat can be used to represent any short-term HRI, indepen-dent from the context. In this way, the model can be pop-ulated with interaction content without any specific knowl-edge of planning.

The main contribution of this paper is the integration ofsuch conditional plans for HRI with tools that make theplans more robust. Specifically, taking the ideas explored inJAMES and adding recovery procedures for circumstancesthat are not contained in the model.

3 System OverviewThe main objective of this work is to provide a frameworkfor the generation of robust conditional plans that can dealwith uncertainty in the initial state. To achieve that, we pro-pose a multi-layered architecture that includes a PlanningSystem, a Plan Translator and a Plan Executor. The over-all architecture is shown in Figure 1. The Planning System(Section 3.1) contains the formulation of the short-term HRIas a conditional planning problem and generates a condi-tional plan using the Contingent-FF planner. The plan is thentranslated into a Petri-Net Plan (Seciton 3.2) and finally it isexecuted by the PNP executor module. The following sub-sections describe the two novel components developed inthis work: the conditional planning system and the transla-tor to PNP. The third component is described in (Ziparo etal. 2011) and available in http://wiki.ros.org/pnp ros.

3.1 Planning SystemThe Planning System is built upon ROSPlan (Cashmore etal. 2015). ROSPlan is a framework for embedding a PDDLplanner into a ROS system. ROSPlan automatically gen-erates a PDDL planning problem, passes this to an exter-nal planner, and dispatches the plan. We have extended theframework to generate conditional planning problems andto make use of a contingent planner, Contingent-FF (Hoff-mann and Brafman 2005b). The plan dispatch module is alsoreplaced, to instead pass the contingent plan entirely to thePNP translator.

We use the formalisation of contingent planning fromHoffmann and Brafman (2005b), where observations anduncertainty are added to (sequential) STRIPS with condi-tional effects.

A planning problem comprises of a domain D describ-ing the actions and problem instance P describing the initialstate and goal condition. This is extended with uncertaintyabout the initial state, non-deterministic action effects, and

Page 3: Short-Term Human-Robot Interaction through Conditional ...iocchi/Teaching/HRI/papers/ICAPS_2017_pape… · Short-Term Human-Robot Interaction through Conditional Planning and Execution

Figure 1: An overview of the system.

observations. The initial state is a belief state represented bya propositional CNF formula I . The possible initial worldstates are those that satisfy that formula. Observations areencoded as special observation actions. Such actions observethe value of a single proposition in the world state, therebysplitting the current belief state at hand and introducing twobranches into the plan. Further details can be found in theoriginal paper (Hoffmann and Brafman 2005b).

In our approach, the domain D is used for modeling thestructure of any short-term interaction, independent from itsspecific context. The problem P characterizes the semanticsof a specific interaction in a given context.

Figure 2 shows a fragment of the PDDL domain forshort-term HRI. As an example, the action display textis used to display a text token to the user. This action isindependent of the content of the text message. Actionsask question and check user response are usedto model a question and answer between the robot and theuser, in which the user response is a proposition that can beobserved.

Figure 3 shows a fragment of the PDDL problem for theTV Show scenario described in Section 4. The tokens (e.g.,t news1, t news2) are entered by the domain expert, andcorrespond to messages specific to the context of the interac-tion. These are used by ROSPlan to automatically generatethe PDDL problem. It is initially unknown which of the newstokens can be given to the user; this is represented using thestandard syntax for uncertainty in the initial state.

Note that, while the planning process considers both thedomain and problem for generating the plan, this division isuseful from the user perspective. The structural representa-tion D requires knowledge about PDDL and planning for-malization, while P can be defined by experts of the appli-cation domain. Although not described in this paper, it ispossible to allow domain experts to model the content of theinteractions without any knowledge of PDDL, by providingthe information in a data format that is then translated in thecorresponding PDDL problem P . In other words, our for-malism integrates the work of a planning expert who definesthe structure of the interaction and of a domain expert whodefines the content of the interactions. The planner is thenable to generate a plan according to the structure and thecontent specified in the model.

The planner Contingent-FF has been embedded in ROS-Plan and used to generate a contingent plan for the interac-

(define (domain hri_contingent)(:requirements :strips :typing)

(:typestokenstatement question - token)

(:predicates(available ?t - token)(responded ?t - token)...)

;; Display text(:action display_text:parameters (?t - statement):precondition (and

(available ?t)...)

...)

;; Ask for something(:action ask_question:parameters (?q - question)...)

;; Check user response(:action check_user_response:parameters (?t - token):precondition (...):observe (responded ?t)

)

;; Wait for screen touched(:action wait_for_hri...)

)

Figure 2: A fragment of a domain for human robot interac-tion.

tion. The plan is parsed as a Directed Acyclic Graph (DAG)and published in its entirety as a single ROS message.

3.2 Plan TranslatorThis module parses the dispatched plan and translates it intoa conditional plan. This process also includes a robustifica-tion phase that makes the produced plans robust, i.e. lessprone to replanning/failures due to uncertainty. The robusti-

Page 4: Short-Term Human-Robot Interaction through Conditional ...iocchi/Teaching/HRI/papers/ICAPS_2017_pape… · Short-Term Human-Robot Interaction through Conditional Planning and Execution

(define (problem tvshow)(:domain hri_contingent)(:objects

t_welcome t_intro t_goodbyet_news1 t_news2 t_news3... - statementq_activity q_news... - question)

(:init(unknown (available t_news1))(unknown (available t_news2))(oneof(available t_news1)(available t_news2))

...)

(:goal (and (given t_goodbye)...))

)

Figure 3: A fragment of a problem generated for humanrobot interaction. The problem instance was used in the TVshow scenario.

fication is implemented using rules that allow to add addi-tional actions and recovery procedures to the current in planin order to increase its robustness to situations not modelledin the planning domain. A similar approach has been de-scribed in (Iocchi et al. 2016).We have defined a convenient format to describe conditionalplans showed in this paper:

λ = t1; ...; tnti = a | < c1?t1 : c2?t2; ...; ck?tk >

In this representation λ is a plan and is composed of a se-quence of terms ti. Each term could be either an atomicaction a or a multiple branch over k sensed propertiesc1, . . . , ck. We assume all the conditions ci to be mutuallyexclusive with each other and that each ci is associated to aterm ti that will be executed when ci is true. These branchesare enclosed between angular brackets (<>) and colons (:)separate the different branches.

Our infrastructure comprehends a module called Trans-lator which is responsible for the translation from a genericplanner output to the described representation. In particular,the planner considered in this work is Contingent-FF as de-scribed in the previous section. This component, however, iseasily extensible to consider also other planners and differ-ent output formalisms.

The algorithm that realizes the translation is describedin Algorithm 1. first() is a function that returns the firstpart of the current plan, which could be either an atomicaction or a branching term. In case of atomic actions,CondP lan addAction() creates a conditional plan stateand links it to one created earlier. In case of branches, wesplit them and recursively construct the sub-trees. The + op-erator represents a concatenation operator that handles dif-ferent situations.

The module responsible for the actual generation of the

Algorithm 1: Conditional Plan translation from a planInput : π: the planOutput: λ: the Conditional Plan corresponding to π

1 Function translCondP lan(π) : λ2 σ = first(π);3 π′ = rest(π);4 if π = 0 then5 return {};6 else7 if σ = ai then8 λ′ = CondP lan addAction(ai);9 else

10 if σ = φ1?π1 : ... : φn?πn then11 Φ = CondP lan splitBranch(σ);12 foreach φi ∈ {φ1, ..., φn} do13 λi = translCondP lan(πi);14 λ′ = CondP lan merge(Φ, λ1, .., λn);

15 return λ′ + translCondP lan(π′);

PNP is called Generator. The algorithm that from the con-ditional plan representation generates the actual PNP is Al-gorithm 2. This algorithm realizes a graph visit on the struc-ture described by the conditional plan. It keeps track of thevisited states and, at each iteration, it checks the possibleoutcomes of the states. If the current state has no actionassociated to it and is not final, it keeps going with recur-sion. Otherwise, there are two possible cases depending onwhether the current state has successors. If there are no suc-cessors, we add a final state to the PNP. If there there aresuccessors, the algorithms processes each of them. For eachsuccessor state that has not been visited yet, the algorithmadds a branch to the PNP and marks it as visited. Otherwise,if one successor state has been already visited, the algorithmadds an edge going back from that state to the current one.

The complexity of the presented algorithms is linear in thenumber of plan states. For the translation algorithm, we gen-erate a conditional plan state for each action in the computedplan. For the generation algorithm, we explore these statesand generate the correspondent PNP. The functions used forthe generation of the conditional plan and the PNP have aconstant cost since they perform only local operations with-out looking at the whole plan.

4 Integration and ExperimentsThe presented system has been fully implemented and testedon a mobile robots in several different demonstrations inpublic environments. The developed software1 is based ontwo existing software libraries: ROSPlan2 and PNP3. Alsothe software described in this paper has been developed un-

1Open-source release will be available in a non-anonymous ver-sion of the paper.

2http://kcl-planning.github.io/ROSPlan/3pnp.dis.uniroma1.it

Page 5: Short-Term Human-Robot Interaction through Conditional ...iocchi/Teaching/HRI/papers/ICAPS_2017_pape… · Short-Term Human-Robot Interaction through Conditional Planning and Execution

Algorithm 2: PNPgen from Conditional PlanInput : π: a Conditional Plan,

SS: a stack containing the places to exploreOutput: πc: PNP generated from a Conditional Plan

1 Function genCondP lan(π) : πc2 s = first(π);3 π′ = rest(π);4 SS ← s;5 while (SS 6= 0) do6 s = top(SS);7 a = action(s);8 if (a = 0 & s 6= sf ) then9 genCondP lan(π′);

10 O = outcomes(s);11 if O = 0 then12 πc = PNP addFinal(s);13 genCondP lan(π′);14 foreach element o of O15 do16 s′ = next(o);17 φ = obs(o);18 if s′ 6∈ SS then19 PNP addBranch(πc, φ);20 SS.push(s′);21 else22 PNP GoBack(πc);

der the ROS4 middleware and it is thus suitable for integra-tion with other robotic components.

More specifically, the developed ROS node for the trans-lation component waits for a plan on the ROS topic pub-lished by the planning system, processes this plan as de-scribed in the previous sections producing a PNP and dis-patches the PNP to the pnp ros module that will execute it.This process can be executed both off-line or on-line duringthe execution of a robot task. In the on-line case, the pnp rosmodule replaces the current plan with any new coming plan.

4.1 Use casesThe proposed system has been tested in different environ-ments characterized by short-term human robot interactionswith non-expert users. The plans defined and generated withthe proposed system have thus been exposed to execution ina real environment. Almost all interactions have been cor-rectly handled, regardless the lack of experience of the usersinteracting with the robot, thus showing the effectiveness ofthe proposed method.

A video showing the interactions that occurred in theseexperiments (with the exception of the third, to preserveanonymity of the authors for the review) is added as addi-tional material to this submission. In all the examples shownin this video, the robot executes short-term interaction plansgenerated with the system described in this paper. While

4www.ros.org

the navigation of the robot in the environment was manu-ally driven (mostly for safety reasons). All the interactionsthat required input from the users (through speech or usingthe GUI on the tablet of the robot) have been modeled andimplemented as sensing actions and the flow of execution ofthe plan is chosen at execution time according to the out-come of such interaction (i.e., sensing actions).

Figure 4: A user interacting with the robot during the Eras-mus welcome day.

display_init;waitfor_personhere;display_text_welcome;ask_whichcontinent;waitfor_continent_@C;ask_whichcountry_@C;waitfor_country_@N;display_text_@N;display_image_@N;ask_wantphoto;waitfor_no;display_text_greet;display_text_goodbye;display_init

Figure 5: The plan for the Erasmus Welcome day.

1. Erasmus welcome day. The system was tested duringa Faculty meeting for Erasmus students. Short-term interac-tions were designed to welcome the students, to ask the stu-dents the from which country they come and to show someinformation about the countries. Since the list of studentsand thus of countries was known in advance, it was possibleto pre-define all of the possible countries and to generate ap-propriate questions and answers for each country. This pro-cess was done by a semi-automatic tool for the generation ofinteractions and is not described in this paper.

Page 6: Short-Term Human-Robot Interaction through Conditional ...iocchi/Teaching/HRI/papers/ICAPS_2017_pape… · Short-Term Human-Robot Interaction through Conditional Planning and Execution

The robustification phase in this example has been imple-mented as a recovery procedure acting when users do notanswer a question after a timeout and restarting the interac-tions in these cases.

About 40 students were present in the event and at leasthalf of them interacted with the robot. Since user studieswere not a focus of this experiment, we have not collectedspecific data about such interactions, but just observed thatall the executions of plans were correctly completed (insome cases, thanks to the recovery procedure).

The Conditional Plan is shown in figure 5. This formatis generated from the DAG by the Translation Module, andthen into a PNP, as described in Section 3.2.

Figure 6: A group of users interacting with the robot atMaker Faire 2016.

2. Maker Faire. In this experiment, the robot was de-ployed during a Maker Faire for three days to interact withpeople; children in particular. The interaction contained dif-ferent choices: information about the robot, quizzes, andjokes. Also in this example, the robustification phase addedsome recovery procedures to take into account some unex-pected behaviors of the users.

During the three days of the event, we needed to changethe content several times. For example, (i) we realized thatsome interactions were too long and we wanted to simplifythem, (ii) we wanted to change some of the content of theinteractions for the day in which schools were present, and(iii) we wanted to test different interaction modalities.

To this end, the use of a planning system was fundamentalin order to generate new plans on the fly when needed, byjust changing the specification of the problem.

During these experiments, the robot interacted with about300 people. Most of the executions were successfully, whilea very few executions failed and required a manual restartof the system. These situations were due to unexpected in-teractions, such as, for example, users closing the windowrunning the interaction program or restarting the operatingsystem on the tablet.

display_init;waitfor_answera;askimg_whichinfoABCD;waitfor_HRIreceived;<answera ?

display_image_info_alfred;display_text_info_alfred:

answerb ?display_image_info_r20;display_text_info_r20:

answerc ?choose_joke;waitfor_joke_@J;display_text_joke_@J:

answerd ?display_image_info_roberto;display_text_info_roberto

>;display_text_goodbye;restart

Figure 7: A conditional plan for the Maker Faire.

display_init;waitfor_start;display_image_coaches;display_text_name;display_text_intro;waitfor_screentouched;display_image_activity;ask_whichactivity;waitfor_HRIreceived;< news ?

display_text_news;ask_whonews;waitfor_answer_@Q;display_image_answer_@Q;display_text_answer_@Q:

joke ?display_text_joke

>;waitfor_screentouched;display_image_coaches;display_text_goodbye;restart

Figure 8: A conditional plan for the TV Show.

An example Conditional Plan is shown in figure 7, in thesame format generated by the Plan Translator module. Inthis plan there exists conditional branches. The sensing ac-tions generated by the planner are not visible in this plan, asthey are instead combined into multiple branches (see Algo-rithm 1 in Section 3.2).

Page 7: Short-Term Human-Robot Interaction through Conditional ...iocchi/Teaching/HRI/papers/ICAPS_2017_pape… · Short-Term Human-Robot Interaction through Conditional Planning and Execution

display_text t_init

wait_for_hri

display_image i_coaches

display_text t_name;

display_text t_intro

wait_for_hri

display_image i_activity

ask_question q_whichactivity

wait_for_hri

check_user_response t_news

display_text t_news

check_user_response t_joke

display_text whonews

wait_for_hri

display_image_answer_@Q

display_text_answer_@Q:

wait_for_hri

display_text_joke

display_image i_coaches;

display_text t_goodbye

Figure 9: The conditional plan generated by Contingent-FFin DOT form.

3. TV Show. The third demonstration described in this pa-per was performed during a live TV show in which the robotwas interviewed alongside a human. Before going to the TVstudio, a sketch of the demo was prepared and presented tothe TV show director one hour before the show. The directormade some comments about the interactions and asked tomodify something and to be flexible with respect to timingof the show. Before the live show, the plan was modified andduring the show the conditional plan possessed the requestedflexibility.

A Conditional Plan is generated by Contingent-FF for thisproblem and converted into a DOT graph, as shown in fig-ure 9. This plan is then translated into the intermediate for-mat shown in figure 8. Note that the extended PDDL used byContingent-FF can only produce binary branches upon sens-ing actions, such as in check user response. These ac-tions do not represent any action taken by the robot, but in-stead signify a branch point in the plan. As a result they donot appear in the translated plan.

5 ConclusionsIn this paper, we described a method and a fully imple-mented robotic system using conditional planning for gen-erating and executing short-term interactions by a robot de-ployed in a public environment.

While conditional planning has previously been used inshort-term HRI, our focus is on 1) the separation of the struc-ture of the planning model and the context of the interaction,2) the integration into an execution environment through theROSPlan and Petri Net Planning frameworks. The systemhas been fully implemented and tested on a real robot in sev-eral different scenarios and open-source release of the code.

In future work we intend to formalize the conditionalplanning domain used for our examples, providing linksbetween the common interaction patterns and the generalstructures in our domain.

ReferencesBonet, B., and Geffner, H. 2000. Planning with incom-plete information as heuristic search in belief space. In Proc.of the Conference on AI Planning and Scheduling, 52–61.AAAI Press.Cashmore, M.; Fox, M.; Long, D.; Magazzeni, D.; Ridder,B.; Carrera, A.; Palomeras, N.; Hurtos, N.; and Carreras, M.2015. Rosplan: Planning in the robot operating system. InProceedings of the 25th International Conference on Auto-mated Planning and Scheduling (ICAPS’15), 333–341.Erdem, E.; Aker, E.; and Patoglu, V. 2012. Answer setprogramming for collaborative housekeeping robotics: rep-resentation, reasoning, and execution. Intel Serv Robotics.Fiore, M.; Clodic, A.; and Alami, R. 2016. On planning andtask achievement modalities for human-robot collaboration.In Experimental Robotics, 293–306. Springer InternationalPublishing.Hanheide, M., et al. 2015. Robot task planning and expla-nation in open and uncertain worlds. Artificial Intelligence.

Page 8: Short-Term Human-Robot Interaction through Conditional ...iocchi/Teaching/HRI/papers/ICAPS_2017_pape… · Short-Term Human-Robot Interaction through Conditional Planning and Execution

Hoffmann, J., and Brafman, R. 2005a. Contingent planningvia heuristic forward search with implicit belief states. InProceedings of the 15th International Conference on Auto-mated Planning and Scheduling (ICAPS-05), 71–80.Hoffmann, J., and Brafman, R. 2005b. Contingent planningvia heuristic forward search with implicit belief states. InProceedings of the 15th International Conference on Auto-mated Planning and Scheduling (ICAPS-05), 71–80.Iocchi, L.; Jeanpierre, L.; Lazaro, M. T.; and Mouaddib,A.-I. 2016. A practical framework for robust decision-theoretic planning and execution for service robots. In Inter-national Conference on Automated Planning and Schedul-ing (ICAPS), 486–494.Lallement, R.; De Silva, L.; and Alami, R. 2014. HATP:An HTN planner for robotics. In 2nd ICAPS Workshop onPlanning and Robotics, 20–27.Little, I., and Thibaux, S. 2007. Probabilistic planning vs.replanning. In In ICAPS Workshop on IPC: Past, Presentand Future.Mcdermott, D.; Ghallab, M.; Howe, A.; Knoblock, C.; Ram,A.; Veloso, M.; Weld, D.; and Wilkins, D. 1998. Pddl - theplanning domain definition language. Technical Report TR-98-003, Yale Center for Computational Vision and Control,.

Milliez, G.; Lallement, R.; Fiore, M.; and Alami, R. 2015.Using Human Knowledge Awareness to Adapt Collabora-tive Plan Generation, Explanation and Monitoring. In Inter-national Conference on Human-Robot Interaction.Petrick, R. P. A., and Bacchus, F. 2004. PKS: Knowledge-based planning with incomplete information and sensing. InProceedings of the System Demonstration session at ICAPS.Petrick, R. P. A., and Foster, M. E. 2013. Planning for so-cial interaction in a robot bartender domain. In Proceedingsof the Twenty-Third International Conference on AutomatedPlanning and Scheduling, ICAPS.Petrick, R. P. A., and Foster, M. E. 2016. Using general-purpose planning for action selection in human-robot inter-action. In Proceedings of the AAAI Fall Symposium on Arti-ficial Intelligence for Human-Robot Interaction (AI-HRI).Tomic, S.; Pecora, F.; and Saffiotti, A. 2014. Too coolfor school - adding social constraints in human aware plan-ning. In Proc. of 9th International Workshop on CognitiveRobotics.Ziparo, V.; Iocchi, L.; Lima, P.; Nardi, D.; and Palamara, P.2011. Petri Net Plans - A framework for collaboration andcoordination in multi-robot systems. Autonomous Agentsand Multi-Agent Systems 23(3):344–383.