MacNeil-LehrerPartnershipMacNeil-Lehrer Productions and the Universityof Southern California have formed a long-termpartnership to pursue research and developmentof Next Generation Journalism technologies.

The project will explore immersive, interactive,three-dimensional audio and video formats andtools for recording, production and transmissionof news and information. The technologies will bedeployed initially for a new daily in-school newsprogram, the.News, now being developed byMacNeil/Lehrer Productions (MLP).

The research will be conducted at the IntegratedMedia Systems Center (IMSC), the U.S. NationalScience Foundation Engineering Research Centerfor multimedia, located at USC’s Viterbi School ofEngineering.

“Journalism, especially for the young, must takeadvantage of emerging technologies” said DanWerner, executive producer for special projects atMLP. “We are extremely pleased by this opportunityto team up with the IMSC which is a leader in

research efforts tomore effectivelytransfer informationin the digital age.”

The project willbuild on recentmilestones in IMSC’sdecade-long researchin immersive media,including its GeoDecfamily of visualization

technologies introduced in 2005 and IMSC’simmersive audio-video Internet transmissions ofsymphony and chamber music concerts.

“We are delighted to work with such adistinguished partner,” said Adam Clayton Powell III,Director of IMSC. “This will combine our laboratory’slongstanding multimedia research with the ViterbiSchool Serious Games Initiative, an investigation ofinteractive media for applications including educa-tion, health care and emergency preparedness.”

The project will be a collaboration among severalengineering disciplines represented at IMSC and thejournalism expertise of USC’s Annenberg School forCommunication.

MacNeil Lehrer Productions produces The NewsHour withJim Lehrer, which airs weeknights on more than 300 localPBS stations, as well as other programs for public, commer-cial and cable television. Other recent television programsproduced by MLP include Do You Speak American?, RobertMacNeil’s journey across America to look at why Americansspeak the way we do; Debating Our Destiny: A History ofPresidential Debate; LadyBird, a profile of LadyBird Johnson.MLP is also the leading force behind the By the People alocal-national journalism-civic engagement project.

Spring 2006Published by the USC Integrated Media Systems Center

IMSC News

Pictured above: Grand Avenue Intervention site framed in yellow.

IMSC Research supported bynew Annenberg Center GrantIMSC has received additional support from the Annenberg Center forCommunication for IMSC GeoDec project research to explore a rangeof applications in urban planning and democratic empowerment.

The new grant, which is in addition to the Annenberg Center’s annual supportfor IMSC research, will fund research to extend GeoDec’s interactive three-dimensional tools to downtown Los Angeles, to depict the area as it nowexists and to support planning of the proposed $1.8 billion redevelopment ofdowntown Los Angeles.

This research project will be conducted in conjunction with the Norman LearCenter at the USC Annenberg School for Communication and the Lear Center’sGrand Avenue Intervention, focusing on a proposed new park for downtownLos Angeles.

IMSC’s GeoDec research includes a family of interactive, three-dimensional,highly accurate visualization tools, from rapid modeling to depiction of livedynamic data, including live video. Led by Professor Cyrus Shahabi, GeoDec ispart of IMSC’s Decision Support research area, which is devoted to research onpresenting massive amounts of data in real time in forms and displays that canbe quickly understood.

“This is precisely the type of cutting edge interdisciplinary project that theAnnenberg Center for Communication should be supporting,” said JonathanAronson, professor and executive director of the Annenberg Center forCommunication, announcing the new grant.

“We are grateful for the recognition of IMSC research in this area,” saidAdam C. Powell III, Director of IMSC. “We are eager to expand the Center’sresearch into new and exciting areas of urban planning and civic engagement.”

“The beauty of these tools,” added Martin Kaplan, associate dean of theUSC Annenberg School and director of the Norman Lear Center, “is that theywill help both citizens and design professionals, continued on page 2

to the Spring edition of the IMSC newsletter!To old friends, welcome back; to newcomers, we hope you return often.

Looking back at 2005, IMSC strengthened its traditional base of industryrelationships in the computer and aerospace industries while expanding ourlaboratory’s initiatives in media, online information, journalism, education andentertainment.

In all of these fields, our research is characterized by its broad and inherentcombination of disciplines. Cinema, Communications, Fine Arts, Journalism andMusic are all integral to IMSC’s initiatives:

For example, our newest partnerships include MacNeil-Lehrer Productions,among the most honored broadcast journalism organizations in the U.S. To pursuea research program for MacNeil-Lehrer — even to design the research agenda —requires an interdisciplinary effort among multiple disciplines of the USC ViterbiSchool of Engineering together with our USC colleagues across the campus,especially in the Annenberg School for Communication.

For our laboratory’s partnership with McKinley Technology High School inWashington, D.C., funded by a supplemental grant from the National ScienceFoundation, IMSC’s Serious Games initiative draws on engineering, games, designand education expertise.

For IMSC’s research extending the laboratory’s GeoDec visualization tools intourban planning and civic engagement applications, IMSC researchers worked withthe Annenberg School for Communication and the Norman Lear Center, supported

by a new grant from the Annenberg Center. And for continuing milestones in IMSC immersive Scalable Immersive Environments and Human Performance Engineering,

faculty from several different fields working with sister laboratories and schools across USC combine their efforts in research thatrises well above the sum of their parts.

IMSC even helped create a new CD, “All Is Bright,” which hit #5 on Billboard magazine’s classical top ten chart over theChristmas season!

Looking ahead to 2006, this will be an exciting and challenging time for IMSC. The only constant will be change — and thatis what makes research interesting.

—Adam C. Powell IIIDirector, IMSC

IMSC News2Page

IMSC’s First CDThe IMSC Immersive Audio Laboratoryparticipated in the recording of a CD thatis now available from AVIE Records. IMSCfaculty Chris Kyriakakis and TomlinsonHolman worked with James Donahue, along-time IMSC collaborator in immersiveaudio and the Principal Recording Engineerat WGBH in Boston. The recording tookplace in Methuen Memorial Music Hall inMethuen, Mass. that houses the largestconcert organ in the US.

Maestro Grant Llewellyn led the Handeland Haydn Society chorus to record acollection of carols from both sides of theAtlantic. Some are well known and otherswere commissioned specifically for thisproject. The Handel and Haydn Society is its191st season and has a long tradition of musicalexcellence. Among its achievements are the American premieres of Handel’s Messiah (1818),Verdi’s Requiem (1878), and Bach’s St. Matthew Passion (1889).

The CD is entitled “All is Bright” and reached reached #5 on the Billboard Classical charts.One of the songs that was commissioned from composer Tom Vignieri has been

processed and rendered for 10.2 Immersive Audio and is available for demonstration inthe IMSC Immersive Audio Lab. Vignieri and Maestro Llewellyn plan to visit IMSC to discussfurther collaborations with IMSC in immersive performance.

no matter where they are located, imaginealternate futures for a real physical space.”

“We are very excited to work withAnnenberg on this project and we see it asan opportunity to extend GeoDec’s utilities tothe Urban Planning application area,” saidCyrus Shahabi of the IMSC.

The Annenberg Center for Communicationis an independent USC center that identifiesand supports cross-disciplinarycommunications research, education anddissemination across the university. Moreat http://www.annenberg.edu/

Based at the USC Annenberg School forCommunication, the Norman Lear Centeris a multidisciplinary research and publicpolicy center exploring implications of theconvergence of entertainment, commerce,and society. More on the Norman LearCenter’s Grand Avenue Intervention projectat http://www.grandintervention.org.

Annenberg Center Grantcontinued from page 1

from the Director

Welcome

The Video Game industry has emergedas the hottest field in interactiveentertainment. Video gamedevelopment is a dynamic, multifaceteddiscipline requiring a diverse blend ofskills, practical experience, current andhistorical knowledge and theoreticalfoundation. Through the diversity ofcourse work, theory and application,including hands-on experience withindustry-standard management anddevelopment tools, this seminar isdesigned to align with students’academic and professional goals.

This summer, IMSC will once againoffer students the opportunity to learnabout video game production at aseminar being offered by Dr. AnthonyBorquez. The IMSC Video Game DesignSeminar will provide students with notonly an overview of the video gameproduction process, but alsofoundation knowledge for gamedesign. In the course, students willgain hands-on experience developingvideo games utilizing various softwareapplications. Upon completion ofthe course, students will be able toconceptualize, design, develop,

implement and integrate current andemergent video game features andtechnologies.

Given it strategic location in southernCalifornia, USC has developedrelationships with various gamecompanies including Electronic Arts,Sony Playstation, Activision, and others.These key relationships have enabledUSC to bring a variety of guest speakersinto the classroom. Over the pastthree years, lead game designers,programmers and artists from avariety of companies have provided

USC students with insight and guidancefor creating compelling games. Ahighlight from the 2005 summer gamedesign program included weekly fieldtrips to the Electronic Arts (EA) campusin Playa Vista where students engagedin Q/A sessions from different EAproduct groups.

For more information on theIntroduction to Video Game DesignSeminar, please contact Dr. AnthonyBorquez at anthonyb@usc.edu fordetails, or visit http://www.usc.edu/dept/admissions/programs/summer/seminars_seminars.shtml.

IMSC research featuredat conferences in AfricaAdam Clayton Powell III, Director of the IntegratedMedia Systems Center, presented the Center’slatest research in communication technology attwo fall conferences in Africa.

Delegates from around the continent attendedand discussed IMSC research on the future ofcommunications technology and information atSeptember’s annual “Highway Africa” conferenceat Rhodes University, in Grahamstown, SouthAfrica. Subjects included immersive audio andvideo, three-dimensional and 360-degreerepresentations of information and entertainment,and two-way haptics for remote healthapplications.

The rapidly falling barriers to entry, bothtechnological and financial, were also discussed,as the presentation touched on content creationand origination in Africa and the Middle East.Examples from South Africa, Egypt and SaudiArabia were reviewed.

In response to the presentation, delegatesdebated how the accelerating pace oftechnological change is affecting the evolutionof formal African media and of unofficialinformation sources across the continent.

The presentation was summarized in an articlewhich was published in the November, 2005, issueof the Rhodes Journalism Review.

In December, IMSC was invited to participate inthe African Media Forum in Nairobi, Kenya, wherethe focus of discussion was the future of globalvideo journalism.

IMSC research that was discussed includedstreaming and synchronization of multiplevideo and audio streams, and the cost-effectivecapturing, storage, and rendering of high-resolution video and immersive audio underdifficult field conditions.

The Nairobi conference attracted more thanthree dozen delegates from Africa, Asia, Australia,Europe and the U.S. Participants included seniormanagers and editors of news organizationsranging from CNN to Al Jazeera, executives ofAfrican media groups from East Africa’s NationMedia Group to South Africa’s SABC and e.tvtelevision networks, and delegates from politicaland technical organizations including the AfricanUnion and Intelsat.

A team from USC has been invited to returnin September for the next “Highway Africa”conference, to participate in and help coordinatethe closing day’s presentations on the future ofcommunications technology on the Africancontinent.

IMSC News 3Page

IMSC Offers an Introductionto Video Game Design Seminar

Summer 2005 IMSC Video Game Design Seminar participants

IMSC: Celebrating 10 Years1996-2006

With a new 3-year NSF grant, IMSC’s HYDRA Rears its Many StreamsRoger Zimmerman, research assistantprofessor in computer science and researcharea director for media immersionenvironments at IMSC, and his team havereceived prestigious NSF support for theirHYDRA project. The grant is part of anNSF program focusing on AdvancingCollaborative and Intelligent Systems andtheir Societal Implications. The focus ofHYDRA — the name loosely represents anacronym for High-performance DataRecording Architecture — is to studyfundamental research questions in the areaof streaming media architectures.

The goal of the IMSC researchers is toinvestigate the challenges inherent in abroad class of innovative applications thatacquire, store, manipulate and disseminatedigital continuous media (CM) streams,such as audio, video, and haptics, inreal-time and on a large scale.

With the widespread deployment ofbroadband networks, streaming media aredriving many existing and new applications.To name just two examples, Voice-over-IP(VoIP) telephony and personal videorecorders (often called “TiVos” after thecompany that pioneered the technology)are now permeating our daily lives.However, most often each application initself is a custom solution that does not“play well with others.”

One of the most glaring examples isthe lack of media integration in the home,where numerous digital media devicesand networks exist such as still cameras,camcorders, media servers, home gateways,broadband and wireless networks.Currently, the seamless manipulation andsharing of data among, for example, TVsaround the house remains elusive.

The research team plans to change thiswith HYDRA by focusing on a unifiedparadigm that integrates multi-streamrecording, retrieval and control in asynergistic manner to provide well-definedservices for all media, cognizant of theircharacteristics such as bandwidthrequirements, resolution and modality.

The results and insights obtained fromthe HYDRA project through fundamentalresearch and testbed evaluations are directlyapplicable to IMSC’s long-term vision ofImmersive Reality, specifically the area ofScalable Immersive Environments (SIE) ledby Professor Shri Narayanan. The aim ofSIE is to enable a world in which people areconnected and empowered by their digitalenvironment. The goal is to develop

technologies thatreproduce audioand video realitywith extremelyhigh fidelity; andare cognizant ofand facilitate theinteraction ofpeople with oneanother whilescaling todifferent devicecapabilitiesranging fromhigh resolution,stationary roomenvironments tomobile, handheldform-factors. Acritical compo-nent of SIE ismultimodal streamprocessing and some of the topics of studyaddressed by Zimmermann and his teaminclude low-latency live streaming, large-scale media storage and retrieval, admissioncontrol algorithms, and protocols enablingsmooth stream acquisition and playback.

At IMSC researchers are pursuing anumber of testbeds that pervasively usestreaming media. These projects are primecandidates to benefit from the HYDRAresearch. One example is the DistributedImmersive Performance (DIP) project,involving a multi-disciplinary group of keyinvestigators. In addition to Zimmermann,the DIP team is comprised of Elaine Chew(ISE), Alexander A. Sawchuk (EE), ChrisKyriakakis (EE) and Christos Papadopoulos(CS). DIP explores one of the mostchallenging scenarios of networkedmedia technology: creating a seamlessenvironment for remote and synchronousmusical collaboration. The objective of DIPis to develop the technology for interactivemusical performances in which theparticipants — subsets of musicians, theconductor and the audience — are indifferent physical locations and areinterconnected by very high fidelitymultichannel audio and video links. In thisenvironment achieving a low end-to-enddelay is extremely crucial to achieve acoherent ensemble performance.

The DIP team has conducted pioneeringexperiments in networked musicalperformance with expert musicians fromthe Thornton School of Music and has useda HYDRA prototype to capture and store

sessions including high-definition video,audio and MIDI streams for time-shiftedrendering and performance analysis.

HYDRA’s capabilities are also crucialto the remote maintenance and trainingoperations that are being studied byPratt & Whitney Institute for CollaborativeEngineering (PWICE), established jointlybetween Korean Air, Pratt & Whitney, InhaUniversity and IMSC. The mission of thisproject is to develop advanced informationprocessing and communication technologiesthat target aviation maintenance andtraining. Here HYDRA’s capability tomanipulate live, interactive streams is ofcritical importance.

HYDRA is enabling high-definition (HD)live streaming over traditional IP networks,such as the Internet, and it forms thebasis of a multi-site, trans-continental HDremote collaboration system with theultimate objective to allow aircraft andengine inspections and repairs with a rapidturn-around time. Such tasks requiretechnicians in multiple locations tocollaborate for problem solving.

Korean Air experts at Incheon airportoffices and with the aircraft on the tarmacin South Korea are interacting with enginetechnicians at Pratt & Whitney’s Hartford,CT, facility. Proving seamless live mediastreaming is very challenging in thisheterogeneous environment where devicesrange from low-power handhelds withwireless connections to sophisticated roominstallations on gigabit networks.

continued on next page

Left to right: Roger Zimmerman, Min Qin, Sakire Arslan Ayand Beomjoo Seo

IMSC News4Page

IMSC News 5Page

A group of students from USC's ViterbiSchool of Engineering and the MarshallSchool of Business were selected to competethis spring in the Rice University BusinessPlan Competition. This three-day competitionis the largest of its kind and is intendedto simulate the real-world process of

young entrepreneurs soliciting start-up fundsfrom early-stage investors and venturecapital firms.

Over 120 schools submitted business plansworldwide and only 36 were selected tocompete at Rice University in Houston, Texas.

Other qualifying business schools from eliteuniversities such as Harvard, MIT, Stanford,and University of Pennsylvania will be com-peting with USC for over $200,000 in prizes.

The USC team is comprised of Ph.Dcandidate Leslie Liu from IMSC, four MarshallMBA students — Daniel Owen, Jackson

Hsieh, Joseph Lin, and KamaleshJha — as well as Tina Hsieh fromUCLA’s Anderson School ofManagement. Leslie Liu has beenpursuing his studies at IMSC forthe past four years, focusing in hisresearch on multimedia streamingarchitectures. He met the otherteammates at the MarshallGraduate Technology Alliance2005 event when he gave apresentation on IMSC’s streamingtechnology.

The team’s business planevolves around a dynamic peer-to-peer Internet streamingsolution that will give consumersthe freedom to control theirmusic and media from anywherethey go. The core technology wasdeveloped under the guidance ofLeslie Liu’s advisor, RogerZimmermann, one of IMSC’s keyinvestigators.

Teams attending the competi-tion have the opportunity tomeet and network with over 100

venture capital principals, earlystage investors, successful entrepre-

neurs and senior business leaders who willserve as judges for the event. The judgesmake decisions on which business venturethey would most likely fund based on boththe technology’s strength and businesspotential.

Joint IMSC/Marshall Team Qualifies for theLargest Business Plan Competition in the U.S.

McKinley Technology High Schoolin Washington, D.C., becameIMSC’s newest K-12 educationpartner this year.

The product of several yearsof research, BioSIGHT’s InteractiveStreaming Storyboard tool alongwith a modest digital library ofImmunology and Cell Biologyanimations, as well asMetalloman, IMSC Serious Gamessoftware for high school biologystudy, have been deployed atMcKinley following evaluationslast year in Los Angeles, with over200 current users at McKinley.

“The integration of theses

products into McKinley’s curriculum and theanticipated benefits to McKinley’s studentsvalidate IMSC’s effort to impact scienceeducation with innovative technology toolsand applications,” said Prof. Wee LingWong of the Serious Games Lab at IMSC.Wong further asserted that “it will enableus to incorporate assessment data andteacher and student experiences in futureresearch.”

Not quite two years old, McKinleyTechnology High School offers studentsintensely focused curricula in bio/medicaltechnology, information technology, andbroadcast technology, together with atraditional liberal arts education. More athttp://mths.k12.dc.us/.

Other areas that are being studied byhis team include industrial monitoring —such as intelligent traffic management —and remote tele-rehabilitation incollaboration with IMSC investigatorMargaret McLaughlin.

“The capabilitiesdeveloped under the HYDRAumbrella have tremendousappeal and we expect themto have a significantpractical impact on a broadrange of applications thatmake use of multimediasensors,” Zimmermannpointed out.

An integral part of theproject is the evaluation ofthe HYDRA design in severalof these domains. Theprototype implementationand experimental researchof this project will also beused as a teaching tool forcourses covering multimediaand server technology.The latest results areavailable from the projectWeb site at http://dmrl.usc.edu/hydra.html.

Left to right: Jackson Hsieh, Kamalesh Jha, Roger Zimmermann, Daniel Owen and Leslie Liu

IMSC Partners with Washington D.C.High School

McKinley High School students usingIMSC’s digital library of science contentanimation.

Creating a system that lets children talk tocomputers instead of using of conventionalmouse or keyboard controls won the 2005IEEE Signal Processing Society “Best Paper”Award for the USC Viterbi School ofEngineering’s Shrikanth Narayanan and acollaborator.

Narayanan, who holds appointmentsin the Viterbi School’s departments ofelectrical engineering and computerscience, as well as the USC Collegedepartment of linguistics, based the paperon research done in 2000-01 with his

co-author, Alexandros Potamianos of theUniversity of Crete.

The award will be presented at theannual meeting of the 16,000-membersociety, to be held this May in Toulouse,France.

The paper, “Creating ConversationalInterfaces for Children,” addresses threeseparate problems.

First, it describes at length the particularproblems of creating systems that canrecognize children’s speech, which isacoustically quite distinct from that ofadults. Children also have much widervariation in their pronunciation of wordsthan do adults, creating additionaldifficulties.

The bottom line was that standardmethods for Automatic Speech Recognition(ASR) did four times worse on children’sspeech than adults. However, specialadjustments made by Narayanan and

Potamianoswere able tobridge the gapand bring theerror ratedown into thestandard adultranges.

But is voicecontrol auseful andeffectivetechnique forchildren? Thenext part ofthe study was

a controlled “Wizard of Oz” setup in whichchildren played a well-known educationalgame (Where in the USA is CarmenSandiego?). Half of the children used thestandard mouse and keyboard techniques.The other half spoke their commands andchoices, which an unseen human observer(“the Wizard”) then executed.

Quizzed afterward on how they liked

playing using voice versus mouse, an over-whelming number loved it — “Ninety-threepercent rated the interface 4 or 5.”

The final element in the paper describeshow the researchers built an interface fora simple game using ASR. The prototypewas a program that prompts children toplay a spelling game, while also casuallyinteracting with them and offering praise.The character was Agent Chimp, and whilethe game was elementary, it was effectivein holding the attention of the eight smallchildren (ages 8-14) who played.

“Overall, the prototype represents asuccessful first effort at building a multi-modal system for children with an emphasison conversational speech,” concludedthe authors at the time.” We expect thedata from such prototypes will helpfurther conversational human-machineinteraction.”

In fact, according to Narayanan, this ishappening: “Some of the work in thispaper serves as a basis for a currentprojects on automated literacy assessment[for young children] funded by NSF, and weare hoping that some of these ideas willbe used in some of the advanced virtuallearning environments that we are tryingto create presently at USC,” he said.

NSF funded the research described inthe paper

The “Best Paper” honor is only the latestdistinction for Narayanan, who was mostrecently named a fellow of the AcousticalSociety of America in November 2005.

Non-ProfitOrganizationUS Postage

PAIDUniversity of

SouthernCalifornia

Integrated Media Systems CenterViterbi School of EngineeringUniversity of Southern CaliforniaCharles Lee Powell Hall, Floor 33737 Watt WayLos Angeles, CA 90089-0274

213-740-2592 Email: brandosb@usc.eduhttp://imsc.usc.edu

Viterbi Voice for Kids Interface Wins IEEE Signals PrizeThree-part project teaches machines to understand children’s speech

“These ideas will beused in some of theadvanced virtuallearning environmentsthat we are tryingto create presentlyat USC.” A

big

ailK

aun