Ruzena Bacjsy Gives a Keynote Talk at Twenty Year Celebration of NSF REU in Computer Vision at UCF

TWENTY YEAR CELEBRATION .............. 1

MINSKY FILLS THE HOUSE! ................. 2

COMPUTER VISION DISTINGUISHED SPEAKER SERIES ............................... 3

DURIP AND PROFIT GRANTS ............ 4

COMPUTER VISION LAB PUBLICATIONS 2007 .......................... 5

#1 DOWNLOAD .................................. 6

2007 GRADUATES ............................ 7

UCF DATASETS ................................ 8

Page 1

project to MIT Ph.D. and to Ohio State Professorship.” As the title indicates, Prof. Davis talked about his journey through the years and how the REU program helped him be prepared. Jay Hackett of Harris Corporation and graduate of the REU program gave a talk on “How REU helped me to become a manager at Harris Corporation.” After lunch, we thought it would be nice to have a poster session where everyone could walk around and see what 2007 REU participants were working on. The students had been working diligently throughout the summer on their REU projects and were able to highlight what they had accomplished in this forum. These undergraduates spoke informally about their research and fielded questions from other attendees.

During the reception, luncheon and banquet attendees were given the opportunity to mingle with old friends and new. At the banquet Professor Ruzena Bajcsy gave a keynote. She talked about computer vision research from her perspective, the work she and her students did and how it related to trends in computer vision. The talk also discussed what she envisioned researchers should learn from her experience and what new futuristic directions she could foresee.

The Computer Vision Lab at the University of Central Florida (UCF) has continued to be a NSF REU (Research Experience for Undergraduates) site since 1987. On August 3rd, 2007, we celebrated the twenty year anniversary by inviting current and previous participants, their advisors, and some distinguished guests. These guests included the Computer Vision Program Director at NSF, Dr. Daniel DeMenthon, as well as President Hitt and Provost Hickey who honored us with their presence and remarks at the celebration’s kick-off. A full day of events was scheduled and included panels, a poster session by current students and a keynote talk at the banquet was given by Professor Ruzena Bacjsy of UC Berkeley, who is also former NSF CISE director. REU celebration started on the evening of August 2, with a dinner party at Shah residence which is close to UCF campus. Most of the attendee were able to attend this exciting dinner party with excellent food, drinks and games. On August 3, after President Hitt kicked off the celebration, Drs. Shah, Bowyer, and Krishnan gave presentations on “The First Decade of REU” that was followed up with a presentation by Dr. Lobo on “The Second Decade of REU” later in the day. Several panel sessions were organized to discuss the “Graduate School after REU” (moderated by Professor Joan Peckham of University of Rhode Island); “How to Start a Startup” (moderated by Dr. Ganapathy Krishnan); “Next Steps in Mentoring Undergraduates in Research” (moderated by Professor Louise Stark of University of Pacific); and “Challenges for New Assistant Professors” (moderated by Professor R. Kasturi). These sessions were aimed at both current and past REU participants in different stages of their career. Professor James Davis of Ohio State University presented a technical talk on Computer Vision entitled “From REU summer

Computer Vision Lab hosted Marvin Minsky on April 1-2. On the first day, Professor Marvin Minsky presented his talk on “The Emotion Machine”. “The Emotion Machine” is also the title of Minsky’s new book. In it, he shows why we should expand our ideas about thinking and how thinking itself might change in the future. Considered by some to be the “Father of Artificial Intelligence”, Minsky’s research has led to both theoretical and practical advances in artificial intelligence, cognitive psychology, neural networks, and the theory of Turing Machines and recursive functions. By far, his presentation had the best turn out of the entire seminar series. The auditorium was filled to the brim and even spilled out the doors into the hallway! In addition to his talk, Professor Minsky and his wife, Gloria, had two busy days which included meeting with the dean, director of EECS, Vice President of Research, luncheon with EECS faculty, and a discussion session with the Computer Vision group.

Minsky Fills the House!

Page 2

Computer Vision Lab University of Central Florida Dr. Mubarak Shah Agere Chair Professor Director, Computer Vision Lab Ms. Cherry Tran Laboratory Manager Research Associates Dr. Pingkun Yan Dr. Jun Xie Ph.D. Students Hassan Abbas Adeel Ali Saad Ali Pavel Babenko Arslan Basharat Alexei Gritai Min Hu Saad Khan Jingen Liu Janaka Liyanage Merrill McKee Omer Orhan Vladimir Reilly Mikel Rodriguez Imran Saleemi Paul Scovanner Anant Suraj Vemuri M.S. Students Yusuf Aytar Phillip Berkowitz Ryan Faircloth Eric Leach Enrique Ortiz

B.S. Students Joan Baldriche Andrew Miller James Schneider Brandyn White

Back row: Janaka Liyanage, Brandyn White, Joan Baldriche, Vladimir Reilly, Paul Scovanner Center Row: Jingen Liu, Saad Masood Khan, Andrew Miller, Saad Ali, Dr. Mubarak Shah, Arslan Basharat, Pavel Babenko, Ramin Mehran Front Row: Fangshi Wang, Bilal Orhan, Suraj Vemuri, Mikel Rodriguez, Yusuf Aytar, Enrique Ortiz, Bulent Tastan

2008 Computer Vision Group

Computer Vision Distinguished Speaker Series

Continued Success in 2007

The Computer Vision Distinguished Speaker Series continued to grow strong in 2007. Professor Ted Adelson from the Massachusetts Institute of Technolofy kicked of the year with his

presentation on “Image Statistics, Visual Perception and Visual Illusions.” Next, Professor David Lowe from the University of British Columbia presented his talk on “Object Recognition from Invariant Local Features.” In it, Lowe gave an overview of the invariant feature approach, as well as some recent

applications such as location recognition and automated stitching of digital images into panoramas. The Co-Director for the Institute for Robotics and Intelligent Systems at the University of Southern California, Professor Gerard Medioni, visited the Computer Vision Lab in March. Medioni presented a talk on “Tensor Voting: Review, Applications to Computer Vision and Machine Learning” in which he briefly reviewed tensor voting, focused on the application of tensor voting to real computer vision problems and showed how tensor voting can be applied to problems in higher dimensions while keeping the computational complexity at reasonable levels. Dr. James Duncan of Yale University visited us in April to present his talk on “Model-Based Biomedical Image Analysis.” In his talk, Duncan reviewed image analysis strategies that make use of models based on geometrical and physical/biomechanical information to help constrain the range of possible solutions in the presence of uncertainties in handling the date related to image acquisition parameters, the variability of normal human anatomy and

physiology, the presence of disease or other abnormal conditions, and a variety of other factors. In September, Dr. Aaron Bobick visited us to present his talk on “Seeing Action.” Bobick is both Professor and Chair of the School of Interactive Computing at the Georgia Institute of Technology. His talk showcased a variety of techniques for the representation and recognition of action. Examples of his work in the areas of perception and cognition, computer vision and artificial intelligence/cognitive science were also presented. The month of October brought Dr. Demitri Terzopoulos who is the Chancellor’s Professor of Computer Science at the University of California, Los Angeles. He presented a talk on “The Simulation of Humans and Lower Animals Applied to Computer Vision” which addressed the emerging discipline that spans the computational and biological sciences and discussed how humans and lower animals

engender novel applications in visual sensor networks and surveillance. Last, but not least, Dr. Larry Davis of the University of Maryland visited us in November. Davis serves as both Professor in the Institute for Advanced Computer Studies and Chair of the Computer Science Department. His research on computer vision problems

associated with detecting people in crowded situations, tracking people through gaps in observation and recognition of safety and security violations in surveillance video were discussed in his talk entitled “Event Recognition in Surveillance Video: The Challenge of Identity Maintenance.” For more information on this series and to view the schedule of upcoming speakers, please visit http://www.eecs.ucf.edu/~vision and click on Distinguished Speakers.

Page 3

Prof. David Lowe

2007 Visiting Scholars Marco Leo

The Italian National Research Council launched a competition for young researchers wishing to participate in research activities to be carried out in prestigious foreign institutions. Dr. Marco Leo, a researcher at the Institute of Intelligent System for Automation in Bari, won a grant within the Council’s “Short Term Mobility Pro-

gram” and sought to carry out his duties with us. Leo arrived in June of 2007 and stayed for 3 weeks. During this stay, Leo was able to work closely with members of the lab and its director and noted that “My visit allowed me to take advantage of their ex-perience and to reorient my projects. Prof. Shah was always will-ing to talk and spent a lot of his precious time to understand my research challenges and to explain to me his valuable opinion.”

Fangshi Wang In April 2007, Dr. Fangshi Wang, a professor of fourteen years at Beijing Jiatotong University wrote to Dr. Mubarak Shah for an opportunity to study and do research with the Computer Vision Lab. Wang’s research focuses on se-mantic annotation and retrieval of images and videos. She believed that “I can improve my research capability and enlarge my knowledge in this field” by studying and researching abroad. Shah agreed with Wang as did her uni-

versity who agreed to pay all travel and living expenses during her one year stay that started in September 2007.

Prof. Gerard Medioni

Page 4

Staff Writer, Chris Cobbs wrote a feature article entitled “Linking Eyes In The Skies” for the front page of the March 22nd Business Section of the Orlando Sentinel. The story was written about Professors Niels da Vitoria Lobo and Mubarak Shah and their work with unmanned aerial vehi-cles (UAV). Shah and Lobo wrote a proposal in response to the Department of Defense’s (DoD) Defense University Research Instrumentation

Program (DURIP). Through this DURIP Competition, funds were awarded via grants for the purchase of research equipment costing $50,000 or more. Lobo & Shah were awarded $210,600. Regarding the fast increment of the number of UAVs used in the battlefield, only little work has been done for researching on multi-ple autonomous UAV framework for visual sensing purposes. Pro-fessors Niels da Vitoria Lobo and Mubarak Shah will put together a

team of multi-platform UAVs—a mixture of fixed-wing and rotary-wing UAV platforms—for studying systemically coordinated visual sensing aerial framework. The Computer Vision Laboratory at UCF has extensive experience in visual tracking with multi-camera framework, aerial mosaic-view construction, automatic target detec-tion/recognition, and vision-guided UAV navigation. The project will fund a new avenue to extend such experience for enabling wide-range survey and accurate close-range reconnaissance by the team of multi-platform UAVs. The technical problems to be solved in-clude: aerial mosaic-view construction, hostile event detection, automatic target recognition, and constructing integrated 3-D view with multiple images from different viewpoints. The equipment will also enable a richer and expanded set of course projects and independent study efforts by undergraduate and graduate students. UCF has a thriving and growing nationally renowned program for undergraduate research experience in computer vision.

DURIP: A Deployable Multi-Platform UAV Team for Visual-Sensing Purposes

$1.2 Million from National Science Foundation PROFIT: Pictures Represent Opportunities For Inspiration in Technology

The National Science Foundation’s Innovative Technology Experi-ences for Students and Teachers (ITEST) program re-sponds to current concerns and projections about short-ages of STEM professionals and information technology workers in the United States and seeks solutions to help ensure the breadth and depth of the STEM workforce. ITEST supports the development, implementation, test-ing and scale-up of models, as well as research studies to address these questions and to find solutions. This par-ticular ITEST, PROFIT, was developed by Niels da Vi-toria Lobo, Juli K. Dixon, Mubarak Shah and Regina H. Gresham. Computing with pictures is an innovative way to introduce secondary school students to Information Technology, so this comprehensive project for students and teachers is centered around introducing Computer Vision and

Imaging experiences into core curriculum of mathematics and IT. The projects are aimed at students in grades 9 and 10. Each year, twenty teachers of mathematics and or IT classes are selected from high schools in the greater Orlando area. The teachers each select three students from the female or minority population in their up-coming class, and these sixty girls and minorities attend a two-week summer youth institute. The teachers receive three weeks of summer professional experience at the beginning of the PROFIT year, are supported 40 hours of follow up contact in monthly weekend activities that include interactions with local IT companies, and attend a 40 hour reflective work-shop at the end of the PROFIT year. The teachers

also teach the students in 80 hour summer institute.

Shah Receives University Distinguished Researcher Award Through use of a peer review procedure consistent with university guidelines and the relevant senate resolutions, the College of Engineering and Computer Science selected Dr. Mubarak Shah to receive the College Award for Excellence in Distinguished Research for 2007. In receiving that award, Shah was automatically nominated for the University Distinguished Research Award. The University Research Council and the Division of Vice President for Research evaluated all applicants’ files for cumulative value and impact of research efforts at UCF within the discipline and to society, recognition of research efforts by the individual’s peers in the same or in related disciplines, publication and presentation of research results, external grant and contract support for the research work appropriate to the candidate’s disci-pline. After reviewing all files, Shah was named the University Distinguished Research for 2007.

CONFERENCES Saad Ali, Arslan Basharat, and Mubarak Shah, “Chaotic Invariants for Human Ac-tion Recognition”, ICCV 2007, Rio de Janeiro, Brazil. Pingkun Yan, Saad M. Khan, and Muba-rak Shah, “3D Model based Object Class Detection in an Arbitrary View”, ICCV 2007, Rio de Janeiro, Brazil. Jingen Liu and Mubarak Shah, “Scene Modeling Using Co-clustering”, ICCV 2007, Rio de Janeiro, Brazil. Saad M. Khan, Pingkun Yan, and Muba-rak Shah, “A Homographic Framework for the Fusion of Multi-view Silhouettes”, ICCV 2007, Rio de Janeiro, Brazil. Paul Scovanner, Saad Ali, Mubarak Shah, “A 3-Dimensional SIFT Descriptor & its Application to Action Recognition”, ACM MM ‘07. Augsburg, Germany. Mikel D. Rodriguez and Mubarak Shah, “Detecting and Segmenting Humans in Crowded Scenes”, ACM MM ‘07. Augs-burg, Germany Javed Ahmed, Mubarak Shah, Andrew Miller, Don Harper, and M.N. Jafri, “A Vision-Based System for a UGV to Han-dle a Road Intersection”, AAAI Twenty-Second Conference on Artificial Intelli-gence, Vancouver, British Columbia, 22-26 July 2007. Vladimir Reilly, Saad Ali, and Mubarak Shah, “Motion and Appearance Contexts for Tracking and Re-Acquiring Targets in Aerial Video”, IEEE CVPR, June 2007. Andrew Miller and Mubarak Shah, “Foreground Segmentation in Surveillance Scenes Containing a Door”, IEEE ICME, Beijing, China 2007. Brandyn White and Mubarak Shah “Automatically Tuning Background Sub-traction Parameters Using Particle Swarm Optimization”, IEEE ICME, Beijing, 2007.

Yusuf Aytar, Omer Bilal Orhan and Muba-rak Shah “Improving Semantic Concept Detection and Retrieval Using Contextual Estimates”, IEEE ICME, Beijing, 2007. Yaser Sheikh, Xin Li, and Mubarak Shah “Trajectory Association across Non-overlapping Moving Cameras in Planar Scenes”, IEEE CVPR, Minneapolis, USA 2007. Yaser Sheikh, Alexei Gritai, and Mubarak Shah “On the Spacetime Geometry of Gali-lean Cameras”, IEEE CVPR, Minneapolis, USA 2007. Yaser Sheikh, Asad Hakeem, and Muba-rak Shah “On the Direct Estimation of the Fundamental Matrix”, IEEE CVPR, Min-neapolis, USA 2007. Saad Ali and Mubarak Shah “A Lagran-gian Particle Dynamics Approach for Crowd Flow Segmentation and Stability Analysis”, IEEE CVPR, Minneapolis, USA 2007. JOURNALS

Paul Smith, Niels da Vitoria, and Mubarak Shah, “Resolving Hand Over Face Occlu-sion”, Image and Vision Computing, Vol-ume 25, Issue 9, 1 September 2007, Pages 1432-1448. Asaad Hakeem and Mubarak Shah, “Learning, Detection and Representation of Multi-Agent Events in Videos”, Artifi-cial Intelligence, 171 (2007) 586–605.

Orkun Alatas, Pingkun Yan, and Mubarak Shah, “Spatiotemporal Regularity Flow (SPREF): Its Estimation and Applications”, IEEE Trans. Circuits and Systems for Video Technol-ogy, vol. 17, No. 5, May 2007, pp. 584-589. Omar Javed, Khurram Shafique and Muba-rak Shah, “Automated Surveillance in Real-istic Scenarios”, IEEE Multi Media, Janu-ary/March 2007.

INVITED TALKS

Mubarak Shah, “The Fundamental Matrix in Human Action Recognition”, Honda Research, Mountain View, CA, April 25, 2007. Mubarak Shah “Visual Analysis of Crowded Scenes”, UCLA, Los Angeles, CA, May 29, 2007.

Mubarak Shah, “Visual Analysis of Crowded Scenes”, UC-Irvine, Irvine, CA, May 30, 2007.

Mubarak Shah, “Overview of Video Sur-veillance and Monitoring Research at UCF”, Hughes Research Lab, Malibu, CA, May 31, 2007.

Mubarak Shah, “Aerial and Video Surveil-lance and Monitoring”, Capstone Talk, IEEE Workshop on Multimodal Sentient Computing: , Minneapolis, MN, June 22, 2007.

Mubarak Shah, “Recognizing Actions, Objects, and Actions as Objects”, Keynote Talk, BMVC 2007, Coventry, England, September 10-13, 2007.

Mubarak Shah, “Video Surveillance and Monitoring”, ICIAP 2007, Modena, Italy, September 10, 2007 Mubarak Shah, “Recognizing Actions, Objects and Actions as Objects”, Keynote talk at British Machine Vision Conference (BMVC), Warwick, UK, September 12, 2007

Mubarak Shah, “Video Surveillance and Monitoring Using Distributed Cameras”, ICDSC 2007, Vienna, Austria, September 25-28, 2007.

Mubarak Shah, “Visual Analysis of Crowded Scenes”, Keynote Talk, ISVC07, Lake Tahoe, NV, November 26-28, 2007. Mubarak Shah, “Visual Analysis of Crowded Scenes”, Keynote talk at Interna-tional Conference on Machine Vision, Islamabad, Pakistan December 28, 2007.

2007 Publications

Page 5

#1 Download

Page 6

Alper Yilmaz and Omar Javed, both previous members of the Computer Vision Lab, co-authored an article with Dr. Mubarak Shah titled “Object Tracking: A Survey.” This article has ranked number one in the top ten most popular ACM magazines and Computing Survey articles in 2007. The article appeared in ACM Computing Surveys, Vol.38, No.4,

December 2006.

NSF Graduate Fellowship Enrique G. Ortiz was awarded the National Science Foundation Graduate Research Fellowship (NSF GRF) for the 2007 Award Year. The NSF GRF is awarded to approximately 1,000 students every year. It is based on a nationally com-petitive fellowship application that is evaluated according to the NSF Merit Review Criteria of Intellectual Merit and Broader Impacts. The fellowship provides funding for a maximum of three years that can be used over a five-year period. Graduate fellows receive a stipend ($30,000 per year), and coverage of tuition costs.

Alper Yilmaz

Undergraduate Student Combines Passion for Vehicles with CV

Omar Javed

Andrew Miller, an undergraduate research student at the Computer Vision Lab, has loved cars and planes all of his life. From building Lego models to programming the controls of the robot that his high school entered in the FIRST competition, Andrew has found ways to combine his enthusiasm for motion with his talent in computer science. At the Computer Vision Lab, Andrew has put those skills to good use. The projects on which he has worked include: ♦ ETISEO surveillance evaluation in Nice, France, 2006. The Computer Vision Lab was one of

five international teams to receive a special mention; Andrew was team leader ♦ CLEAR surveillance and aerial video evaluation in Baltimore, Md., 2007. Andrew was primary

author of two summary papers that will be published. ♦ ICME 2007 conference in Beijing. He presented paper on analysis of surveillance video of opening doors. ♦ AAAI 2007 conference in Vancouver, 2007. The Computer Vision Lab presented a paper on a robot car in an intersection; Andrew was second author and made the oral presentation. ♦ At the ICRA 2008 conference to be held in Pasa-dena, California, in May 2008, Andrew will make an oral presentation on landing a simulated UAV. In addition , Andrew was the only undergraduate mem-ber of the UCF team that entered a robotic car in the DARPA Urban Challenge, in which the UCF team reached the finals. In fact, he was the one who drove the car from Florida to California. He now is working on his undergraduate Honors Thesis in his major (Computer Science) on UAV navigation. Andrew’s future plans are

to pursue a doctorate in computer science. “I’m interested in the intersection of robotics and computer science,” he says. “I love the hands-on work of building the actual vehicles, so my work at the Computer Vision Lab has been very satisfying.” Andrew will doing his internship at Texas Instruments in Dallas during summer of 2008.

Andrew Miller with DARPA director Tony Tether and the rest of UCF team in California at the DARPA Grand Challenge National Qualification Event.

Andrew Miller

On December 21, 2007 Dr. Mubarak Shah was informed that the International Society for Optical Engineering (SPIE) Board of Directors and Fellows Committee had elected him to the grade of Fellow of SPIE. According to Brian Culshaw, SPIE President, “The annual recognition of Fellows provides an opportunity for us to acknowledge outstanding Members for their service to the general optics community.” This honor was bestowed upon Shah for his contribution to Video Understanding and Shape from Shading in Machine Vision.

In order to be eligible, “...one must be a regular Member of the Society on October 1 of the year they are nominated, and has become distin-guished through outstanding contributions in the relevant technologies.” Candidates who meet the eligibility requirements, and are nominated for promotion, will be evaluated by the Society’s Fellow Committee on the nominee’s technical accomplishment, service to the general optical community, and service to SPIE.

Shah will be recognized at the SPIE Defense and Security Symposium in Orlando that runs from March 16 – 20, 2008.

SPIE Elects Shah as a Fellow

Computer Vision Lab 2007 Graduates

Page 7

Alexei Gritai In August 2007, Alexei Gritai graduated from the University of Central Florida with his Ph.D in Computer Science. Gritai’s dissertation was entitled Multi-view Geomet-ric Constraints for Human Action Recognition and Tracking. This dissertation made three major contribu-tions to automatic analysis of human actions. First, he conjectured that the relation-ship between body joints of two actors in the same pos-ture can be described by a 3D rigid transformation. Second, he proposed a novel projection model for cam-

eras moving at a constant velocity in 3D space, Galilean cameras, and derive the Galilean fundamental matrix and apply it to human action recognition. Third, he proposed a novel use for the invariant ratio of areas under an affine transformation and utilizing the epipo-lar geometry between two cameras for 2D model-based tracking of human body joints.

Asaad Hakeem Asaad Hakeem joined the Computer Vision Lab in 2002 and graduated with his Ph.D. in May 2007. During his time here, he has written twelve publications, including two ICPR papers and one AI Journa l , worked on six different research projects, was nominated for ACM Best Paper Award in 2005 and won the O u t s t a n d i n g Dissertation Award. The world that we live in is a complex network

of agents and their interactions which we term events. Also, the agents can act independently as well as collectively to perform events. With the dawn of the new millennium, the low-level vision tasks such as segmentation, object classification, and tracking have become fairly robust. But a representational gap still exists between low-level measurements and high-level understanding of video sequences. Hakeem’s dissertation on Learning, Detection, Representation, Indexing and Retrieval of Multi-Agent Events in Videos is an effort to bridge that gap where he proposed novel approaches to learning, detection, representation, indexing and retrieval of multi-agent events in videos. Currently, Hakeem works for ObjectVideo in Reston, VA.

Alexei’s Graduation Party

Computer Vision Lab School of Electrical Engineering & Computer Science University of Central Florida

4000 Central Florida Blvd. Orlando, FL 32816-2362

UCF Datasets Sports Action Dataset

♦ Sport actions: diving, golf swing, kicking, lifting, skating, horseback riding, etc from broadcast TV.

http://server.cs.ucf.edu/~vision/projects/action_mach/ucf_sports_actions.rar

Feature Film Action Dataset

♦ “Kissing'' and “Hitting/slapping” actions. http://rodriguez.mikel.googlepages.com/slapsandkissesdataset

Crowds Dataset

♦ Crowds and other high density objects.

http://server.cs.ucf.edu/~vision/people/SaadAli/UCF_CrowdsDataset.zip