50 | NewScientist | 25 December 2010/1 January 2011

BRET

T RY

DER

The droids are trundling into the limelight

Wherefore art thou robot?

HEATHER,” says the performer, “help me with my stylish scarf.” He regally flings his arms in the air, and waits. The woman

he is addressing, Heather Knight, places a black lacy garment around his neck. He brings his arms back down and starts to tell a joke.

The performer, whose name is Data, is rehearsing for his stand-up-comedy debut. Yet Data is not just any comedian: he is a half-metre-tall humanoid robot. He and Knight are doing their thing in Washington Square Park, New York, in front of a crowd of passers-by.

Data is not the first robot to take to the stage in the hope of entertaining an audience (see “Performance a bit mechanical”, page 52). But now there is a bigger goal at stake for performers like him: to acquire acting skills that could help the rest of robotkind interact more fluidly with humans.

And boy, do robots have a lot to learn. Present-day robots can often be annoying, creepy or just plain rude in their interactions with us. Humans are adept at coping with social nuances and subtle cues in communication, says Knight, a PhD student at Carnegie Mellon University (CMU) in Pittsburgh, Pennsylvania, and founder of a New York City-based robotics company called Marilyn Monrobot. “Yet robots today are socially disabled,” she says. For their developers, it’s about more than politeness. Unless robots develop more social awareness, they cannot become more sophisticated, and we will have to wave goodbye to the idea of asking a friendly robot for help with the chores, or any kind of realistic companionship.

Seeking inspiration, robotics researchers have begun testing their robots in the theatre. “People are realising that we have a lot to learn from the arts,” says Leila Takayama of Willow Garage, a robotics company based in Menlo Park, California. She co-organised a workshop on robots and the arts at the Human-Robot

Interaction 2010 conference in Osaka, Japan.Theatre directors and actors can often

provide insights into human interaction that have eluded roboticists, according to Hiroshi Ishiguro, who leads the Intelligent Robotics Laboratory at Osaka University. Robot programmers often don’t know where to start when recreating the speech and myriad movements of a human. Yet theatre directors often know these cues intuitively: they tease the right responses from actors all the time. The theatre is also a great place to test robots’ skills, because the dialogue is scripted and the characters behave predictably, unlike in the real world. Ishiguro has already staged several robot plays in the hope of distilling the elements that could make robots seem more human.

One of the first people to use theatre to polish up robotic social skills was Guy Hoffman. In 2005, he was a PhD student at the Massachusetts Institute of Technology’s Media Lab, working on machines with artificial intelligence. In his spare time, he started taking drama lessons. There he discovered that actors were routinely tackling many of the headaches he faced in his day job, such as making sure that every piece of speech and every action was consistent with an overall personality.

Hoffman decided to put one of his artificial creatures on stage: a robot called AUR, an expressive desk lamp. By flexing its neck AUR can make “eye contact” using its multicoloured light, and narrow its aperture rather like the iris in the human eye. The result was a performance in Cambridge, Massachusetts, in 2007 involving AUR and human actors.

Hoffman focused on making AUR’s acting balance timeliness with spontaneity. He programmed AUR to use the play’s script to predict how the human actors might be

“

” The robot actor was weirdly adorable. I felt surprisingly protective of it”

101225_F_Xmas_05_BotTheatr.indd 50 16/12/10 16:55:53

25 December 2010/1 January 2011 | NewScientist | 51

>

behaving at any moment. It meant AUR could respond in a relatively flexible fashion as well as promptly, without processing delays. AUR could also carry on acting as the other actors delivered their lines. Previous robot thespians have been able to act in a similarly expressive way, but the performance felt more canned because it had to be totally pre-programmed.

Hoffman’s approach seemed to work. The actors and the director said that after a while it felt as if AUR was a character in his own right. “The robot was weirdly adorable. I felt surprisingly protective of it,” reported one actor.

Hoffman reckons timely responses from robots are essential for honing their ability to interact fluidly with humans. Offstage, he conducted an experiment in which he asked 33 people to collaborate on a task with AUR. Those that worked with AUR in his “anticipatory” mode subsequently attributed more human characteristics and intelligence to the robot than those who partnered with him while his predictive skills were switched off. And that’s despite the fact that AUR made more response errors when he was anticipating. “People accept robots taking some risk with their decision as long as they are snappy,” says Hoffman, now at the Interdisciplinary Center Herzliya in Israel. “They kind of forgive the robot for being wrong.” He has since used the same principles to design another robot performer – a marimba player called Shimon, who can improvise jazz together with a human pianist.

Audience reactionBoth AUR and Shimon bounce off fellow performers to polish their social skills, but the response of an audience is just as handy for social training. When Data the robot comedian performed in December at the TEDWomen conference in Washington DC, he started with material selected in a pre-programmed manner from a database of around 200 jokes, says Knight. But he was also listening for laughter, clapping and chatter via a microphone pointed at the audience. Using software that Knight built with colleagues at CMU, the robot then began to pick gags that were more likely to get a laugh.

The jokes were classified according to their theme, degree of interactivity with the audience and other characteristics. If people found risque jokes a turn-off, but liked having Data ask them questions, he would respond to suit their taste, every so often throwing in a random joke to keep the performance fresh.

101225_F_Xmas_05_BotTheatr.indd 51 16/12/10 16:39:18

He also picked jokes according to where he was in his routine, saving his best jokes for the finale, for example.

Knight already has plans to extend these techniques to robots outside the theatrical setting. She will soon use the same software for a robotic tour guide at her university that will personalise its route around the campus and propose activities that it thinks guests will appreciate.

Merely predicting what people will enjoy

52 | NewScientist | 25 December 2010/1 January 2011

Data (above)Roles: stand-up comic in New York, August 2010, and on stage at PopTech in october 2010, and at TeDWomen, December 2010

Helicopter fairies Role: Flying fairies with rotors in A Midsummer Night’s Dream, performed at Texas A&M University in 2009“The robot fairies capture the attention of everyone and don’t let go.” thebatt.com

Wakamaru (right)Role: Acts as a male and female robot living with two humans in the play Hatakaru Watashi (I, Worker), staged in 2008 at osaka University, Japan“Soon they may be signing autographs or trying to roll away from paparazzi”. BBC News

aurRole: Desk lamp in a play called The Confessor, performed in 2007 in Cambridge, Massachusetts“It’s interesting how much meaning a change of color can channel. When he turned and switched to red I instantly got the feeling he wanted something.” thinkartificial.org

GeminoiD-f (below left)Roles: Appears in the play Sayonara (Goodbye), reading a poem to a dying woman, played by a human actor. It was performed in November 2010 in Tokyo“The android, seated on a chair throughout the performance, resembles a part-Russian, part-Japanese woman. Her voice was calm, but her performance a bit mechanical.” Reuters

‘PeRFoRMANCe A BIT MeCHANICAl’

won’t give Data a full set of social smarts, though. To communicate with humans convincingly, automatons must also be able to inject a dash of personality into their actions and words. So for the next stage of Knight’s robo-comedian endeavour, she plans to vary many more aspects of its behaviour besides joke selection.

The plan is for a robot to perform the same joke, or even the same script, to different audiences. While the words will be the same,

all the non-verbal behaviour – gestures, plus the volume and mode of delivery of the robot’s speech – will vary. The idea is to let the software deduce, from the audience reactions, which combinations of non-verbal communication work for which parts of the script. To provide feedback, the audience will wear wristbands that interpret their excitement levels, their gaze will be watched by cameras, and they will be seated in chairs that can detect fidgeting.

For example, a robot playing Juliet might learn by trial and error that when it delivers the line “parting is such sweet sorrow”, it is appropriate to look into Romeo’s eyes and pause before sending him away. A robot insensitive to the crowd’s reaction might instead “blithely stare into the audience and brusquely end the scene”, Knight says.

robot personasKnight hopes to use what she learns from this audience feedback to create a range of believable robot personalities. That would allow you to choose a robot persona to suit a given task. “Personality becomes one of the design principles,” she says.

“I think it’s a great idea – it makes sense,” says Takayama. She likens Knight’s approach to the way a director can tweak a movie based on focus-group screenings. “It’s pretty cool that you could do that with robots.” She cautions, however, that using theatre in this way has its limitations too: an audience may react differently to somebody interacting one-on-one with the robot. “It might not directly translate,” she says. “Knowing it’s on stage, where you are the fourth wall, is different to having one in front of you in a small room.”

Still, Knight points out that this is all part of a rich tradition of placing robots in unexpected settings. For example, making robots play soccer has driven the development of algorithms that can coordinate many robots simultaneously. There was also the DARPA Grand Challenge, a long-distance robot car race that honed the technology of autonomous vehicle navigation.

Back in New York, Data has his audience captivated. He begins to tell another joke. “Waiter! Waiter! What’s this robot doing in my soup?” he says. The punchline: “It looks like he’s performing human tasks twice as well, because he knows no fear or pain.” Not surprisingly, that one falls flat. But Data learns his lesson, and launches into another gag. n

Celeste Biever wonders if she would be funnier if she were a robotYO

SHIK

AZU

TSU

NO

/AFP

/GeT

TY

KRI

S KR

üG

101225_F_Xmas_05_BotTheatr.indd 52 16/12/10 16:39:32