View
224
Download
0
Category
Tags:
Preview:
Citation preview
Introduction to Mobile Sensing with Smartphones
Uichin LeeApril 22, 2013
iPhone 4 - Sensors
Applications
• Transportation– Traffic conditions (MIT VTrack, Nokia/Berkeley
Mobile Millennium) • Social Networking– Sensing presence (Dartmouth CenceMe)
• Environmental Monitoring– Measuring pollution (UCLA PIER)
• Health and Well Being– Promoting personal fitness (UbiFit Garden)
Citysense
MacroSense
CabSense
Eco-system Players
• Multiple vendors– Apple AppStore– Google Play (Android Market)– Microsoft Mobile Marketplace
• Developers– Startups– Academia– Small Research laboratories– Individuals
• Critical mass of users
Scale of Mobile Sensing
Sensing Paradigm
• Participatory: active sensor data collection by users– Example: managing garbage cans by taking photos – Advantages: supports complex operations– Challenges:
• Quality of data is dependent on participants
• Opportunistic: automated sensor data collection– Example: collecting location traces from users– Advantages: lowers burden placed on the user– Challenges:
• Technically hard to build – people underutilized• Phone context problem (dynamic environments)
SENSE
LEARN
INFORM, SHARE AND PERSUASION
Mobile Sensing Architecture
Mobile Computing Cloud
Sense
• Programmability– Managing smartphone sensors with system APIs– Challenges: fine-grained control of sensors, portability
• Continuous sensing– Resource demanding (e.g., computation, battery)– Energy efficient algorithms– Trade-off between accuracy and energy consumption
• Phone context– Dynamic environments affect sensor data quality– Some solutions:
• Collaborative multi-phone inference• Admission controls for removing noisy data
SENSE
LEARN
INFORM, SHARE, PERSUASION
Learn
• Integrating sensor data– Data mining and statistical analysis
• Learning algorithms – Supervised: data are hand-labeled (e.g., cooking, driving)– Semi-supervised: some of the data are labeled– Unsupervised: none of the data are labeled
• Human behavior and context modeling• Activity classification• Mobility pattern analysis (place logging)• Noise mapping in urban environments
SENSE
LEARN
INFORM, SHARE, PERSUASION
Learn: Scaling Models
• Scaling model to everyday uses – Dynamic environments; personal differences – Large scale deployment (e.g., millions of people)
• Models must be adaptive and incorporate people into the process
• If possible, exploit social networks (community guided learning) to improve data classification and solutions
• Challenges:– Lack of common machine learning toolkits for smartphones– Lack of large-scale public data sets – Lack of public repository for sharing data sets, code, and tools
SENSE
LEARN
INFORM, SHARE, PERSUASION
Inform, Share, Persuasion
• Sharing– Data visualization, community awareness, and social networks
• Personalized services– Profile user preferences, recommendations, persuasion
• Persuasive technology – systems that provide tailored feedback with the goal of changing user’s behavior– Motivation to change human behavior (e.g., healthcare,
environmental awareness)– Methods: games, competitions, goal setting– Interdisciplinary research combining behavioral and social
psychology with computer science
SENSE
LEARN
INFORM, SHARE, PERSUASION
Privacy Issues
• Respecting the privacy of the user is the most fundamental responsibility of a mobile sensing system
• Current solutions– Cryptography– Privacy-preserving data mining– Processing data locally versus cloud services– Group sensing applications is based on user
membership and/or trust relationships
Privacy Issues
• Reconstruction type attacks– Reverse engineering collected data to obtain invasive
information • Second-hand smoke problem– How can the privacy of third parties be effectively protected
when other people wearing sensors are nearby?– How can mismatched privacy policies be managed when
two different people are close enough to each other for their sensors to collect information?
• Stronger techniques for protecting people’s privacy are needed
Understanding Smartphone Sensors:accelerometer, compass, gyroscope,
location, etc
Smart Phone/Pad Sensors
GalaxyNexus iPhone4 iPhone5 Samsung
Galaxy S3SamsungGalaxy S4
Galaxy Tab/ iPad2
Accelerometer O O O O O O
Magnetometer (Compass) O O O O O O
Gyroscope O O O O O O
Light O O O O O O
Proximity O O O O O O
Camera O O O O O O
Voice O O O O O O
Pressure (Barometer) O O O
Humidity/IR/Temperature O
Smartphone Sensors
• Accelerometer• Magnetometer (digital compass)• Gyroscope• Light• Pressure (Barometer)• Proximity• Camera• Voice• Temperature/Humidity/IR Gesture (Galaxy S4)
Android APIs
• Package: android.hardware• Classes:– SensorManager – android service– Sensor – specific sensor– SensorEvent – specific event of the sensor = data
SensorManager
• Most sensors interfaced through SensorManager or LocationManager – Obtain pointer to android service using
Context.getSystemService(name)
– For name, use constant defined by Context class • SENSOR_SERVICE for SensorManager • LOCATION_SERVICE for LocationManager
• Check for available sensors using List<Sensor> getSensorList(int type)
– Type constants provided in Sensor class documentation
SensorManager
• Use getDefaultSensor(int type) to get a pointer to the default sensor for a particular type Sensor accel =
sensorManager.getDefaultSensor( Sensor.TYPE_ACCELEROMETER);
• Register for updates of sensor values using registerListener(SensorEventListener, Sensor, rate)
– Rate is an int, using one of the following 4 constants • SENSOR_DELAY_NORMAL (delay: 200ms)• SENSOR_DELAY_UI (delay: 60ms)• SENSOR_DELAY_GAME (delay: 20ms)• SENSOR_DELAY_FASTEST (delay: 0ms)
– Use the lowest rate necessary to reduce power usage• Registration will power up sensor:
mSensorService.enableSensor(l, name, handle, delay);
SensorManager
• Unregister for sensor events using unregisterListener(SensorEventListener, Sensor) or unregisterListener(SensorEventListener)
• Undregistering will power down sensors:mSensorService.enableSensor(l, name, handle, SENSOR_DISABLE)
• Perform register in OnResume() and unregister in OnPause() to prevent using resources while your activity is not visible
• SensorListener is deprecated, use SensorEventListener instead – See documentation for Sensor, SensorManager, SensorEvent and
SensorEventListener
SensorEventListener
• Must implement two methods onAccuracyChanged(Sensor sensor, int accuracy) onSensorChanged(SensorEvent event)
• SensorEvent – int accuracy – Sensor sensor – long timestamp • Time in nanoseconds at which event happened
– float[] values • Length and content of values depends on sensor type
API – Setup
public class MainActivity extends Activity implements SensorEventListener {..
private SensorManager sm = null;…
public void onCreate(Bundle savedInstanceState) {..
sm = (SensorManager) getSystemService(SENSOR_SERVICE);}
protected void onResume() {..List<Sensor> typedSensors = sm.getSensorList(Sensor.TYPE_LIGHT);// also: TYPE_ALLif (typedSensors == null || typedSensors.size() <= 0) … error…sm.registerListener(this, typedSensors.get(0),
SensorManager.SENSOR_DELAY_GAME);// Rates: SENSOR_DELAY_FASTEST, SENSOR_DELAY_GAME,// SENSOR_DELAY_NORMAL, SENSOR_DELAY_UI
}}
API – Processing Events
It is recommended not to update UI directly!
public class MainActivity extends Activity implements SensorEventListener {..private float currentValue;private long lastUpdate;…public void onSensorChanged(SensorEvent event) {
currentValue = event.values[0];lastUpdate = event.timestamp;
}..
}
API – Cleanup
public class MainActivity extends Activity implements SensorEventListener {…
protected void onPause() {…
sm.unregisterListener(this);}
…
protected void onStop() {…
sm.unregisterListener(this);}
..}
Accelerometer
Mass on spring
Gravity Free Fall Linear Acceleration Linear Accelerationplus gravity1g = 9.8m/s2
-1g1g
Accelerometer
STMicroelectronics STM331DLH three-axis accelerometer (iPhone4)
Accelerometer
• Sensor.TYPE_ACCELEROMETER• Values[3] = m/s2, measure the acceleration
applied to the phone minus the force of gravity (x, y, z)– GRAVITY_EARTH, GRAVITY_JUPITER,– GRAVITY_MARS, GRAVITY_MERCURY,– GRAVITY_MOON, GRAVITY_NEPTUNE
Compass
• Magnetic field sensor (magnetometer)
ZX
Y
XY
Z
3-Axis Compass?
Magnetic inclination
Horizontal
Gravity
Magneticfield
vector
Magnetic declination
Magneticnorth
Geographicnorth
Compass
Hall Effect 3-Axis Compass
Compass
• Sensor.TYPE_MAGNETIC_FIELD• values[3] = in micro-Tesla (uT), magnetic field
in the X, Y and Z axis• SensorManager’s constants– MAGNETIC_FIELD_EARTH_MAX: 60.0– MAGNETIC_FIELD_EARTH_MIN: 30.0
Orientation Sensor• Sensor.TYPE_ORIENTATION
– A fictitious sensor: orientation is acquired by using accelerometer and compass • Deprecated
– Now use getOrientation (float[] R, float[] result)• Values[3] – (Azimuth, Pitch, Roll)
– Azimuth, rotation around the Z axis• 0 <= azimuth <= 360, 0 = North, 90 = East, 180 = South, 270 = West
– Pitch, rotation around the X axis• -180 <= pitch <= 180• 0 = sunny side up• 180, -180 = up side down• -90 = head up (perpendicular) • 90 = head down (perpendicular)
– Roll, rotation around the Y axis• -90<=roll <= 90• Positive values when the z-axis moves toward the x-axis.
Orientation: Why Both Sensors?
• We need two vectors to fix its orientation! (gravity and magnetic field vectors)
Tutorial: http://cache.freescale.com/files/sensors/doc/app_note/AN4248.pdf
Gyroscope
• Angular velocity sensor– Coriolis effect – “fictitious force” that acts upon a freely moving object as
observed from a rotating frame of reference
Gyroscope
Gyroscope
• Sensor.TYPE_GYROSCOPE• Measure the angular velocity of a device• Detect all rotations, but few phones have it– iPhone4, iPad2, Samsung Galaxy S, Nexus S– Values[] – iPhone 4 gives radians/sec, and makes it
possible to get the rotation matrix
Accelerometer vs. Gyroscope
• Accelerometer– Senses linear movement, but worse rotations, good for tilt
detection, – Does not know difference between gravity and linear
movement• Shaking, jitter can be filtered out, but the delay is added
• Gyroscope– Measure all types of rotation– Not movement– Does not amplify hand jitter
• A+G = both rotation and movement tracking possible
How to Use the Data – Example
float[] matrixR = new float[9];float[] matrixI = new float[9];
SensorManager.getRotationMatrix(matrixR, matrixI,matrixAccelerometer, matrixMagnetic);
float[] lookingDir = MyMath3D.matrixMultiply(matrixR,new float[] {0.0f, 0.0f, -1.0f}, 3);float[] topDir = MyMath3D.matrixMultiply(matrixR,new float[] {1.0f, 0.0f, 0.0f}, 3);GLU.gluLookAt(gl,0.4f * lookingDir[0], 0.4f * lookingDir[1], 0.4f * lookingDir[2],lookingDir[0], lookingDir[1], lookingDir[2],topDir[0], topDir[1], topDir[2]);
Accelerometer Noise - Simple
• Exponential moving average
const float kFilteringFactor = 0.1f; //play with this value until satisfied
float accel[3]; // previous iteration
//acceleration.x,.y,.z is the input from the sensoraccel[0] = acceleration.x * kFilteringFactor + accel[0] * (1.0f - kFilteringFactor);accel[1] = acceleration.y * kFilteringFactor + accel[1] * (1.0f - kFilteringFactor);accel[2] = acceleration.z * kFilteringFactor + accel[2] * (1.0f - kFilteringFactor);
result.x = acceleration.x - accel[0];result.y = acceleration.y - accel[1];result.z = acceleration.z - accel[2];
Return result;
Accelerometer Noise – Notes
• If it is too slow to adapt to sudden change in position, do more rapid changes when angle(accel, acceleration) is bigger
• You can throw away single values that are way out of average.
• |acc| does not have to be equal to |g| !• Kalaman filters – too complicated?
Other sensors
• Light sensor• Proximity sensor• Pressure sensor• Temperature/Humidity/IR Gesture (Galaxy S4)
Light sensor
• Sensor.TYPE_LIGHT• values[0] = ambient light level in SI lux units• SensorManager’s constants
– LIGHT_CLOUDY: 100– LIGHT_FULLMOON: 0.25– LIGHT_NO_MOON: 0.001– LIGHT_OVERCAST: 10000.0 (cloudy)– LIGHT_SHADE: 20000.0– LIGHT_SUNLIGHT: 110000.0– LIGHT_SUNLIGHT_MAX: 120000.0– LIGHT_SUNRISE: 400.0
Proximity sensor
• Sensor.TYPE_PROXIMITY• values[0]: Proximity sensor distance measured
in centimeters (sometimes binary near-far)
Temperature sensor
• Sensor.TYPE_TEMPERATURE• values[0] = temperature
Pressure sensor
• Sensor.TYPE_PRESSURE• values[0] = pressure• no constants
Recommended