Statement     May,  2014  

TUCKER  BALCH,  ASSOCIATE  PROFESSOR  SCHOOL  OF  INTERACTIVE  COMPUTING,  COLLEGE  OF  COMPUTING  

GEORGIA  INSTITUTE  OF  TECHNOLOGY      Research  on  robot  teams  Beginning  with   Tucker’s   Ph.D.   research   at  Georgia   Tech  with  Ron  Arkin  much  of   his  work   has  focused   on   behavior-­‐based   robotics,   especially   for   robot   teams.   Prof.   Balch’s   work   has  addressed   some  of   the  most   important  questions   facing   the   field  of   cooperative   robotics.  We  enumerate  some  of  the  most  important  work  in  terms  of  those  questions  below:    Q:  How  much  communication  is  enough?  Tucker   addressed   this   question   with   Ron   Arkin   in   the   paper   “Communication   in   reactive  multiagent  robotic  systems.”  They  showed  that  just  a  single  bit  of  communication  to  indicate  a  primitive  signal  such  as  “I  see  food”  conveys  substantial  value.  More  complex  messages  such  as  “I  see  food  at  location  XY”  provide  only  incrementally  additional  benefit.  This  paper  was  also  one  of  the  first  to  show  how  robots  can  communicate  through  the  environment  by  changing  it.  This  paper   with   Arkin   appeared   in   the   first   issue   of   Autonomous   Robots   and   has   been   cited   570  times.    Q:  How  can  robots  move  together  effectively?    Cooperative  movement  in  specific  formations  is  important  in  applications  such   as   tactical   aircraft   activities   and   Army   ground   formations.  Accordingly,  it  is  also  important  for  robot  teams.  The  challenge  is  how  to  provide   control   that   enables   robots   to   maintain   formation   while   also  attending  to  other  important  considerations  such  as  obstacle  avoidance.    Ron   Arkin   and   Tucker   developed   a   behavior-­‐based   approach   to   this  problem  that  was  demonstrated  on  DARPA  vehicles  (figure  at  right).  This  work   was   among   the   first   to   address   this   problem.     It   was   reported   in    IEEE   Transactions   on   Robotics   and   Automation.     It   has   been   cited   2055  times.    

 Q:  How  can  lots  of  robots  move  together  effectively?  The  behavior-­‐based  approach  to  formation  control  Prof.  Balch  developed  with   Arkin  works  well   for   a   team,   namely   2   to   4   robots,   but   it  was   not  clear   how   it   might   scale   to   large   numbers   of   robots.   The   solution,  developed   with   Maria   Hybinette,   was   to   borrow   from   chemistry:   Treat  formation   control   as   something   like   crystal   formation:   Robots   are  attracted   to   specific   “bonding   sites”   defined   by   the   locations   of   their  neighbors.  This  work  was  presented  at  ICRA  and  has  been  cited  293  times.    Q:   How   should   robots   move   cooperatively   to   gather   information   as   a  team?    The  methods  described  above  for  cooperative  robot  movement  provided  effective   formation   control,   but   to   some   extent   they   lacked   a   purpose.    Why  should  robots  be  moving   in  formation?     In  work  with  his   first  Ph.D.  student,   Ashley   Stroupe,   Tucker   began   to   investigate   purposeful  movement  for  teams  in  the  context  of   information  gain.   If  robots  should  

Statement     May,  2014  

be   moving   to   observe   something,   how   could   they   do   that   most   effectively?   Ashley   and   Dr.  developed  an  approach  to  this  problem  whereby  each  robot  would  consider  where  it’s  partners  would   move   to   maximize   information,   then   it   would   move   in   the   best   direction   to   add  information  for  the  team.  The  result  is  coordinated,  purposeful  motion  of  the  team.  An  example  of  the  trajectories  a  team  of  robots  might  follow  using  this  approach  is  illustrated  on  the  right.  Tucker’s  papers  with  Ashley  on  this  topic  have  been  cited  more  than  300  times  in  total.  

 Q:  How  can  pervasive  networks  assist  robots  in  team  tasks?  In   2005   Daniel   Walker,   Keith   O’Hara   and   Balch   invented   the   “Gnat”:   A   small   (about   1.5”  diameter)   communication   and   computation   node   that   could   be  manufactured   at   scale.     They  manufactured  100s  of  Gnats,  and  investigated  how  they  could  be  used  to  solve  team  tasks.  

                   

 The  idea  is  that  Gnats  would  be  distributed  into  the  environment,  perhaps  from  the  air,  and  that  they  would  prepare  an  enhanced  environment  for  mobile  robot  teams  to  interact.  The  Gnats  are  able   to   self   organize   into   communication   networks   that   reflect   the   topology   of   their  environment.  Balch’s  group  demonstrated  how  robot  teams  could  solve  navigational,  coverage  and  foraging  tasks  more  effectively.  All  together,  this  work  has  been  cited  about  100  times.    From  social  robots  to  social  animals  More  recently  Dr.  Balch  has  been  aiming  at  a  different  core  research  question:      Q:  How  can  we  describe  the  behavior  of  social  animals?  This  may  seem  like  a  significant  departure  from  his  earlier  work,  but  as  we  outline  this  research  you  will  see  that  it  is  not.    The  central  idea  is  to  assume  that  the  animals  we  observe  are  acting  according  to  a  robot  program.  Once  we  make  that  assumption  we  can  apply  a  wealth  of  formal  tools   the   task  of   inferring  what   that  program   is   (e.g.,  HMMs,  POMDPs,  etc.).     The  assumption  that  animals  act  according  to  a  behavior-­‐based  program  is  not  too  radical  if  one  considers  that  one   of   the   leading   approaches   to   robot   programming,   behavior-­‐based   control,   originates   in  models  of  behavior  developed  by  biologists.      Why  the  answer  to  this  question  important  Biologists,   psychologists   and   others   seek  models   of   behavior   to   explain   their   subjects.    Most  models  created   in   this   context  are  only  partly   functional  or  partly  descriptive.     For   instance,  a  researcher  might   infer   a   differential   equation   that   predicts   an   aspect   of   behavior   such   as   the  number  of  food  items  collected  over  time.    On  the  other  hand,  the  models  Tucker  is  interested  in  are  complete  explanations  of  behavior,  because  they  describe  the  full  behavioral  “loop”  from  sensing  to  acting.    Other  types  of  models  are  not  complete  in  this  way.    

Statement     May,  2014  

It   is   in   the   context   of   many,   many   agents   that   this   work   is   distinguished   from   others.     The  models   his   systems   learn   are   for   individual   agents,   but   the  models   account   for   behavior   that  arises  through  interaction.    Thus  these  executable  models  (e.g.,  robot  programs)  are  falsifiable  hypotheses   of   system   behavior,   because   they   can   be   tested   in   simulation   with   thousands   of  agents.      Executable  models  are  also  transferrable.  Suppose,  for   instance,  that  we  were  able  to  build  an  algorithmic  description  (robot  program)  of  individual  animal  behavior,  that  when  applied  across  a  many  agents  enables  an  entire  insect  colony  to  solve  an  important  problem.    We  could  adapt  that  result  for  robots,  or  human  decision-­‐making  tasks.      This  work  has  been  carried  out  in  3  main  stages:      

1. Track  the  animals  with  sensors.  2. Build  executable  models  of  their  behavior.  3. Assess  accuracy  of  the  models  in  simulation.  

 Each  of  these  steps  is  a  research  topic  on  its  own.  We  outline  them  individually  below.      Multi-­‐target  tracking  Tracking  animals   in  video   is  a  hard  problem.    For  Tucker’s  work,   the  tracking  problem  is  made  more  challenging  because  we  must  track  many  animals  at  once.  Two  key  challenges  arise  here:  1)   Identification   (who   is  who)  and  2)  Maintaining   separate,   correct   tracks   for  animals   through  interaction  and  occlusion  events.    Dr.  Balch’s  work  in  this  area  began  when  he  was  a  postdoc  at  CMU.    James  Bruce,  Manuela  Veloso  and  Tucker  co-­‐authored  a  paper  on  CMVision,  a  fast  color-­‐based   image   segmentation   algorithm.   The   paper   contributed   two  novel   ideas:  First  a  reformulation  of  color  classification  as  a  table   look  up,  and  second,  a  very   fast   “blob”   finder  based  on   run   length  encoding.    This  work  became  the  de  facto  standard  library  for  color-­‐based  segmentation  in  several  communities  including  RoboCup.  It  is  also  part  of  the  ROS  release  (a  well   known   open   source   robot   operating   system).     The   image   at   right  illustrates   how   an   original   image   (left)   is   segmented   into   colored   regions  (right)  by  CMVision.  This  paper  has  been  cited  646  times    

Color   segmentation   can   only   serve   as   a   tracking   algorithm   if   each  target   is   individually   marked.   This   obviously   limits   its   application.    So,  after  Tucker  moved  to  Georgia  Tech,  he  worked  with  Prof.  Frank  Dellaert   and   student   Zia   Khan   to   create   true  multi-­‐target   trackers.    They   devised   probabilistic   descriptions   of   the   problem,   and   then  translated  these  into  particle  filter-­‐based  trackers.    The  image  at  left  shows  one  of  their  results  in  ant  tracking.    The  work  with  Frank  and  Zia   resulted   in   6   papers   at   top   robotics   and   computer   vision  conferences   (ECCV,   IROS),   and   two   separate   papers   in   the   journal  

PAMI   (highest   impact   factor  of   journals   in   computer   science).    All   together   these  papers  have  been  cited  more  than  1200  times.    

Statement     May,  2014  

These   trackers   focused  on  maintaining  correct  association   through  occlusion,  but   they  did  not  solve  the  identification  problem.  Tucker’s  Ph.D.  student  Adam  Feldman  worked  on  a  solution  to  the  combined  ID  and  track  association  problem  in  his  dissertation  work.    They  created  a  system  that  forms  “tracklets”  or  short  track  segments  of   individual  agents  with  high  confidence.    They  then  combine  that  information  with  data  from  low  spatial  resolution  RFID  tags  to  combine  the  tracklets  into  full  tracks  with  correct  IDs.      More  recently,   in  work   funded  by  the  National   Institutes  of  Health  with  Kim  Wallen  at  Emory,  Dr.   Balch   has   developed   a   novel   tracking   system   for   captive   animals   based   on   active   radio  frequency  ID  tags  (see  figure  below  left).  It  is  the  first  system  we  know  of  that  can  track  dozens  

of  animals    in  real  time  over  several  weeks  in  all  weather  conditions  (above  center).  This  work  is  recent,  but  already  the  team  has  been  able  to  use  the  tracking  data  to  infer  social  relationships  between  individual  monkeys  and  represent  that  data  as  social  tie  strength  graphs  (above  right).  The  monkey   tracking   work   is   in   collaboration   with   Kim  Wallen   at   Emory,   Dan  Walker,   Aaron  Bobick,   Jim   Rehg,   and   Irfan   Essa   at   GT,   and   students   Brian   Hrolenok,   Hanuma  Malladi,   Rahul  Sawhney,  Pipei  Huang,  and  Michael  Novitzky.    Next  steps   in   tracking:  Prof.  Balch   is  collaborating  with   Jim  Rehg   to  develop  his  algorithms   for  tracking  animals   in  outdoor  environments   into  a   scalable  multi-­‐target   tracker.     This  work   is   in  support  of  an  NSF  award  focused  on  understanding  social  animal  behavior.      Executable  models  of  behavior  The  objective  for  Tucker’s  work  in  this  area  is  to  create  descriptions  of  behavior   that   can  be  executed   in   simulation  or  on  a   robot.     The  best  example  of  this  work   is  a  project  with  student  Richard  Roberts  (right).        They   used   instance-­‐based   learning   (KNN)   to   train   a   robot   to   solve  navigation  problems  such  as  obstacle  avoidance,  path  following,  and  even  running  a  slalom  course  (pictured).    The  training  takes  place  as  a   human   drives   the   robot   through   example   courses.   The   robot  “observes”  how  the  human  responds  to  the  navigational  challenges.    The  robot  in  essence  learns  a  model  of  the  human’s  driving  behavior  that  can  be  executed  later.  

 This  same  approach  can  be  used  to  enable  robots  to  learn  the  behavior  of  social  animals.    Tucker  and  his  team  have  explored  this  by   learning  models   of   simulated   agents,   then   validating   the   system   behavior   in  simulation.    The  image  at  left  illustrates  the  performance  of  simulated  agents  performing  a  foraging  task  that  was  learned  from  observation.    

Statement     May,  2014  

Q:  Can  we  learn  switching  hybrid  controllers  from  observation?    In   recent   work  with   his   student   Brian   Hrolenok,   Tucker   has   been   able   to   automatically   learn  executable   models   of   schooling   fish   behavior   and   foraging   ant   behavior.   An   important  contribution  of  this  work  is  that  the  models  include  controllers  for  individual  control  modes,  and  the   rules   for   switching   between   controllers.   The   models   are   acquired   using   tracks   of   animal  movement  (with  tracking  methods  described  above),  and  learned  using  a  variation  of  the  Baum  Welch  algorithm.    Broader  Impact:  Personal  robots  for  CS  education  In   addition   to   his   research   in   robotics   and   social   animal   systems,   Tucker   has   led   several  initiatives  in  education.    Starting   in   2008,   Tucker   led   a   team   of   Georgia   Tech   researchers   in  collaboration  with  educators  at  Bryn  Mawr  College   in   the  creation  of  a  curriculum,   software,   textbook   and   robot   hardware   to   teach  introductory   computer   science.   The   work   was   initially   funded   by  Microsoft  and  then  the  NSF.  Collaborators   included  Prof.  Mark  Guzdial,  Dr.  Jay  Summet  and  Daniel  Walker  at  Georgia  Tech  and  Prof.  Doug  Blank  and  Prof.  Deepak  Kumar  at  Bryn  Mawr.    Dr.   Balch’s   primary   contribution   to   this   effort   centered  on   the  development  of   the   robot   and  software  for  controlling   it,  as  well  as  developing  a  program  for  teaching  the  course  at  Georgia  Tech.     The   robot   and   curricula   were   initially   tested   in   Georgia   Tech’s   CS   1301:   Intro   to  Computing,  and  are  now  a  permanent  part  of  that  course  (above,  right)    CS   1301,   using   these   robots,   is   taught   to  more   than   350   students   each   semester   at   Georgia  Tech.  Overall,   about   2,000  Georgia   Tech   students   have  been   introduced   to  Computer   Science  with  these  robots.  5,000  robots  have  been  distributed  to  students  at  30  institutions.    Broader  Impact:  Massive  Online  Open  Course  (MOOC)  on  Machine  Learning  in  Finance  Tucker  has  recently  initiated  teaching  and  research  in  the  application  of  Machine  Learning   to  Finance.  As  part  of   that  work,  he   teaches  a  course,  CS  7646:  Machine  Learning  for  Trading  on  campus  at  Georgia  Tech.      In   2012,   Tucker   converted   the   first   portion   of   that   course   into   a  MOOC   that  was   offered   via  Coursera.  This  course  is  arguably  the  most  successful  of  Georgia  Tech’s  MOOCs,  with  more  than  100,000  students  enrolling,  and  5,000  completing  the  course.      Prof.  Balch  has   taken  advantage  of  his   experience   in   this  new  domain   to  gather,   analyze,   and  report  on  data  and  techniques  to  help  future  MOOC  instructors  and  teachers.  He  have  published  this   information   as   articles   on   a   public   blog   followed   by   thousands   of   viewers  (http://augmentedtrader.wordpress.com).