Upload
kathryn-moody
View
218
Download
2
Tags:
Embed Size (px)
Citation preview
Virtual Contact Sheets
Wolfgang RichterKiryong Ha, Alok Shankar, Ardalan Amiri Sani*, Jan
Harkes, Lin Zhong*, Mahadev Satyanarayanan
PARALLEL DATA LABORATORYCarnegie Mellon University
*Rice University
Crowdsourcing the News [1]
“When every cell phone can record video and take pictures, everyone is a potential news source.”
Wolfgang Richter © Apr 20, 2023http://www.pdl.cmu.edu/ 2
Who killedIan Tomlinson?
CNN: Caught Red Handed [2]
Wolfgang Richter © Apr 20, 2023http://www.pdl.cmu.edu/ 3
I’m a sneaky thief.I’m a sneaky thief.
Wolfgang Richter © Apr 20, 2023http://www.pdl.cmu.edu/ 4
Problem
CloudCloud
Recent,not uploaded
Older,uploaded
Fundamental Mobile Challenges
• Real-time sync not possible:
• Energy• Infinite battery life desired
• Bandwidth• Increasingly metered and charged
Wolfgang Richter © Apr 20, 2023http://www.pdl.cmu.edu/ 5
How far can we degrade the objects in a search before it loses meaning?
How far can we degrade the objects in a search before it loses meaning?
Just-in-Time Search1. Check virtual contact sheet
• Collection of fidelity-reduced images in the cloud
2. Pull full fidelity image on hit• Search full fidelity; present full fidelity results
to searcher
Wolfgang Richter © Apr 20, 2023http://www.pdl.cmu.edu/ 6
CloudCloud
Wolfgang Richter © Apr 20, 2023http://www.pdl.cmu.edu/ 7
Near Real-Time Sync(1) Search hit(2) Retrieve
Contact Sheet Analogy
Wolfgang Richter © Apr 20, 2023http://www.pdl.cmu.edu/ 8
Evaluating Feasibility [3]
Wolfgang Richter © Apr 20, 2023http://www.pdl.cmu.edu/ 9
Face Detection, RGB Histogram, Brightness Filter
Face Detection, RGB Histogram, Brightness Filter
Benchmark
UsingVarying Resolution and JPEG Quality Factor
Energy Savings
Wolfgang Richter © Apr 20, 2023http://www.pdl.cmu.edu/ 10
10 images =5 seconds call time (Droid)
10 images =5 seconds call time (Droid)
Bandwidth Savings
Wolfgang Richter © Apr 20, 2023http://www.pdl.cmu.edu/ 11
Orders of magnitude savings in bandwidth.Orders of magnitude
savings in bandwidth.
What about Search Accuracy?
Wolfgang Richter © Apr 20, 2023http://www.pdl.cmu.edu/ 12
No one-size-fits-all sweet spot.No one-size-fits-all sweet spot.
Face Detection
Conclusion & Future Work• Searches retain meaning
• No one-size-fits-all, but
• Orders of magnitude bandwidth savings
• Significant energy savings
• More search primitives?• Perceptual hashing• Face recognition• Texture Filters
• Better fidelity reduction?• Keep faces at high fidelity
Wolfgang Richter © Apr 20, 2023http://www.pdl.cmu.edu/ 13
Further Reading[1] TED. http://www.ted.com/talks/paul_lewis_crowdsourcing_the_news.html.
[2] CNN. http://www.cnn.com/2010/CRIME/08/24/new.jersey.theft.photo/index.html?hpt=C1.
[3] SimpleCV. http://simplecv.org/.
Jan Harkes, Mahadev Satyanarayanan, Ardalan Amiri Sani, and Benjamin Gilbert. A Contact Sheet Approach to Searching Untagged Images on Smartphones. CMU Technical Report CMU-CS-11-132.
Ardalan Amiri Sani, Wolfgang Richter, Xuan Bao, Trevor Narayan, Mahadev Satyanarayanan, Lin Zhong, and Romit Roy Choudhury. Opportunistic Content Search of Smartphones. Rice University Technical Report TR0627-2011.
Xuan Bao, Trevor Narayan, Ardalan Amiri Sani, Wolfgang Richter, Romit Roy Choudhury, Lin Zhong, and Mahadev Satyanarayanan. The Case for Context-Aware Compression. HotMobile 2011.
Jason Flinn and Mahadev Satyanarayanan. Energy-aware Adaptation for Mobile Applications. SOSP 1999.
Y. Nimmagadda, K. Kumar, and Y. H. Lu. Energy-efficient image compression in mobile devices for wireless transmission. ICME 2009.
[1] TED. http://www.ted.com/talks/paul_lewis_crowdsourcing_the_news.html.
[2] CNN. http://www.cnn.com/2010/CRIME/08/24/new.jersey.theft.photo/index.html?hpt=C1.
[3] SimpleCV. http://simplecv.org/.
Jan Harkes, Mahadev Satyanarayanan, Ardalan Amiri Sani, and Benjamin Gilbert. A Contact Sheet Approach to Searching Untagged Images on Smartphones. CMU Technical Report CMU-CS-11-132.
Ardalan Amiri Sani, Wolfgang Richter, Xuan Bao, Trevor Narayan, Mahadev Satyanarayanan, Lin Zhong, and Romit Roy Choudhury. Opportunistic Content Search of Smartphones. Rice University Technical Report TR0627-2011.
Xuan Bao, Trevor Narayan, Ardalan Amiri Sani, Wolfgang Richter, Romit Roy Choudhury, Lin Zhong, and Mahadev Satyanarayanan. The Case for Context-Aware Compression. HotMobile 2011.
Jason Flinn and Mahadev Satyanarayanan. Energy-aware Adaptation for Mobile Applications. SOSP 1999.
Y. Nimmagadda, K. Kumar, and Y. H. Lu. Energy-efficient image compression in mobile devices for wireless transmission. ICME 2009.
Wolfgang Richter © Apr 20, 2023http://www.pdl.cmu.edu/ 14
Full Fidelity
Wolfgang Richter © Apr 20, 2023http://www.pdl.cmu.edu/ 15
12,000+ Flickr dataset
Average photo size: 374 KBAverage photo size: 374 KB
Reduce Resolution
Wolfgang Richter © Apr 20, 2023http://www.pdl.cmu.edu/ 16
12,000+ Flickr dataset
Average photo size: 221 KBAverage photo size: 221 KB
JPEG Quality Factor
Wolfgang Richter © Apr 20, 2023http://www.pdl.cmu.edu/ 17
12,000+ Flickr dataset
Average photo size: 6 KBAverage photo size: 6 KB
Bandwidth Savings Table
Wolfgang Richter © Apr 20, 2023http://www.pdl.cmu.edu/ 18
Resolution 100 95 75 50 25 5
640x480 221 KB 114 KB 47 KB 31 KB 19 KB 6 KB
320x240 61 KB 33 KB 14 KB 10 KB 6 KB 2 KB
160x120 17 KB 10 KB 5 KB 3 KB 2 KB 826 B
80x60 5 KB 3 KB 2 KB 1 KB 892 B 460 B
40x30 2 KB 1 KB 725 B 588 B 484 B 345 B
20x20 1000 B 727 B 501 B 436 B 385 B 316 B