TherML: Occupancy Prediction for Thermostat Control Christian Koehler, Brian D. Ziebart, Jennifer Mankoff, Anind K. Dey

Carnegie Mellon University, 5000 Forbes Ave, Pittsburgh PA, 15213 ckoehler@andrew.cmu.edu, bziebart@uic.edu, jmankoff@cs.cmu.edu, anind@cs.cmu.edu

ABSTRACT Reducing the large energy consumption of temperature regulation systems is a challenge for researchers and practitioners alike. In this paper, we explore and compare two common types of solutions: A manual systems that encourages reduced energy use, and an intelligent automatic control system. We deployed an eco-feedback system with the ability to remotely control one’s thermostat to ten participants for three months. Participants appreciated the ability to remotely control the thermostat, and controlled their heating system with 78.8% accuracy, a 6.3% improvement over not having this system. However, despite having feedback and remote control, they still wasted a lot of energy heating when away from home for the day. Using data from our deployment, we developed TherML, an occupancy prediction algorithm that uses GPS data from a user’s smartphone to automatically control the indoor temperature of a home with 92.1% accuracy. We compare TherML to other state-of-the-art techniques, and show that the higher accuracy of our approach optimizes both energy usage and user comfort. We end with recommendations for a mixed initiative system that leverages aspects of both the manual and automated approaches that can better match heating control to users’ routines and preferences.

Author Keywords Machine learning, prediction, heating, location, energy

ACM Classification Keywords H.m. Information Systems: Miscellaneous.

INTRODUCTION Technological advancements over the past few decades have allowed us to live more comfortable lives, but as a result we consume increased amounts of energy. Temperature regulation systems, which are responsible for over 47% of the consumption in an average American household, are a prime target for reducing energy use [23]. Today, most temperature control is performed manually based on habits (such as turning down the temperature when going to bed) and driven by issues of comfort (such as deciding to turn up the heat in the winter when cold). Devices such as programmable thermostats are capable of

decreasing the energy use of a temperature regulation system by reducing the maintained temperatures during particular time periods, such as when homes are empty during the workday. However, only 43.5% of U.S. households own such a system and out of them only 53% reduce the temperature during daytime when nobody is at home [23]. Widespread misconceptions also exist regarding suitable home temperatures. For instance, a 2007 interview study [13] reported that 41% of interviewees believed that room temperatures should be lower in summer than in winter, which leads to high energy use in both seasons.

Two primary strategies can be used to reduce the energy use of temperature regulation systems. Manual techniques typically work to convince users to adopt energy-saving behaviors (e.g., [9]). For example eco-feedback technologies can reduce residential energy use by as much as 20% [8]. However, the problem may not be only motivation but also ease of use. For example, 47% of households that own a programmable thermostat do not use it [20] due to usability breakdowns [5,14]. Additionally, even programmable thermostats do not automatically adjust to changes in behaviors over time, and most are not remotely controllable. In contrast, automatic techniques can adapt to changes in user behavior, and may be as effective as eco-feedback technologies [16,20]. However, they cannot reach optimal efficiency without information about what users find acceptable with respect to comfort. Comfort, rather than temperature, is crucial to acceptance, and it is a complex and “highly negotiable socio-cultural construct” [3] which varies from user to user. Both approaches are in need of further exploration.

To address these issues, we present two contributions. The first contribution is a study of a feedback system combined with a remotely controllable thermostat focusing on user behavior related to temperature control. We deployed our system with ten participants for an average of ninety days. We interviewed participants, used experience sampling whenever the remote interface was used, measured indoor temperature, and gathered GPS data to trace their presence or absence from their home. We found that users increased their accuracy in controlling their heating system by as much as 6.3% over a baseline condition. We explore users’ approach to manual control and the tradeoffs they make between comfort and energy savings.

Our second contribution, therefore, is an automated technique, TherML that is 92.1% accurate at controlling indoor temperature assuming that the goal is to both reduce energy use and to have a fixed, comfortable temperature whenever the user is home. Our algorithm leverages

Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and that copiesbear this notice and the full citation on the first page. Copyrights forcomponents of this work owned by others than ACM must be honored.Abstracting with credit is permitted. To copy otherwise, or republish, topost on servers or to redistribute to lists, requires prior specific permissionand/or a fee. Request permissions from Permissions@acm.org. UbiComp '13, September 08 – 12, 2013, Zurich, Switzerland. Copyright 2013 ACM 978-1-4503-1770-2/13/09…$15.00.

Session: Home Heating UbiComp’13, September 8–12, 2013, Zurich, Switzerland

103

mpPthTathtcoPimTpscbLb(caTps

Aawaoim

MWathsrsnm

machine-learninprediction [25]PreHeat [20] hermostat [18]

TherML can noalso adjust to che home teemperature an

controlled theron the predictiPreHeat [20] anmplementation

TherML. As uperformance bsignificantly (ucomparison of before, we alsoLostComfort (fbefore the use(heating whencompare TherMand find that boThermostat by preferred indoosaving more en

After presentinanalyzing our awe discuss the and argue for aof moments wmpact of users

MANUAL THERWe provided application thahermostat from

showed an estirecommendatiosettings. We dnext, to positivmethods such a

Alias GA

Jane (P02) F

Liana (P03) F

Mary (P04) F

Peter (P05) M

John (P06) M

Kerry (P07) F

Suresh (P08) M

Ram (P09) M

Vibhu (P10) M

Shauna (P11) F

ng technique, based on sm

and the c], by using paot only predict hanges in the d

emperature md user location

rmostat deployion accuracy nd TherML. Wn of Preheat [2user drive timetween TherM

up to 7% for othese two algoo present a comfailing to reacer reaches homn the user is ML and PreHeoth predictive aallowing a m

or temperaturenergy.

ng our user sability to autosorts of errors

a combined apphen comfort i

s forgetting to t

RMOSTAT COusers with

at gave them m any locationmate of their e

ons for energydesigned the vely reinforce as goal setting a

Gen/ Age

# Room- mates

F/26 1

F/29 0

F/57 0

M/24 1

M/28 1

F/38 1

M/25 4

M/25 2

M/25 2

F/41 2

s for path martphone GPS

ommercially ast and current

a user’s returndaily routine, a

more effectiven data from ouyment to simulof two predic

When simulated20] is 1.5% lees increase, th

ML and PreHeur participants

orithms has notmparison alonch a comfortame) and Unn

not home). eat with GPS-algorithms outuch higher dev

e while the use

study of manumatically contthat occurred iproach that cans not an issueturn down the h

ONTROL STUDan Android the ability t

n. In addition,energy consumy-efficient hophone applicabehavior chanand real-time f

Table 1: Demog Preferred Home Temp

20°C

23°C

21°C

20°C

20°C

20°C

20.5°C

20.5°C

20°C

21°C

and destinatiodata. Similar available Ne

t location tracen home time, band then regulaely. We usr manual remolated the impa

ction algorithmd on our data, oess accurate thhe difference eat can increas). Since a diret been conduct

ng the metrics able temperatuecessaryHeatinIn addition w-Thermostat [9tperformed GPviation from ther is away, th

ual control antrol temperaturin each approan take advantage and reduce thheat.

DY mobile phon

to control the, the applicatio

mption and madme temperatuation, describ

nge using knowfeedback [11].

graphics and keCommute

Time In

12 Co

25 Do

9 Gr

11 Ro

18 Wi

8 Bo

tem

12 Aw

ask

10 Us

Co

16 Co

31 Ha

on to

est es, but ate ed

ote act

ms: our an in

ase ect ed of

ure ng we 9], S-he

hus

nd re, ch ge he

ne eir on de

ure ed

wn

ApplicOur Amorninthe useonboarabout tmorninto infltimely

The fira Recoset thewhile the mapreferrthe colhome complicompliand grvalues.applicacompe

Finally

ey facts about pnterpersonal / St

oordinates comfort

oesn’t like being to

rew up with 55 at n

oommate supportiv

ife supportive of lo

oyfriend left therm

mperatures and trie

way for long hours

ked to use a space

sed space heater in

oordinated with roo

oordinates with roo

as 2 children, but p

FRecom

cation designAndroid applicng, at a time sper of the prevird LED and vthe impact of ngs, prior to reluence the useand appropria

rst thing the usommendation Se temperature wat home to 18

anual conditionred temperaturlor of the scree

temperatures iance with thiance with eithreen indicates. In addition ation also sugensate for cool

y, the applic

participants. Getructural Facto

t with roommate

oo cold. Unpredict

night, comfortable

ve of lowering tem

owering temperatu

mostat alone, even w

ed lowering

s. Coordinates with

heater

n very cold bedroo

ommates with vary

ommates, but easil

parent determined

Figure 1: Yellowmmendation Sc

cation has thrpecified by theous day’s cons

vibrating. By stheir behavior

egular departurer’s behavior

ate fashion.

ser sees when oScreen. This scwhile away fro8°C [1,5]. Ourn is based on thre (see Table 1en indicates w

low enoughe recommend

her the away orcompliance w

to temperaturggests wearingtemperatures a

cation calculat

en=Gender ors

table schedule

e trying colder tem

mperatures

ures

when home first. G

h roommates, but

om.

rying schedules

ly chilled roomma

temperature.

w creen

Figuestim

ree main scree user, the phosumption by flshowing this inr on energy usres to work, outo be more g

opening the appcreen encouragom home to 15r quantitative ahe set away an). As shown in

whether the useh. Red indic

dations, yellowr home recommwith both recore recommendg additional cat home.

tes the “env

mperatures

Grew up with low

easily chilled room

ate asked to use a s

ure 2: Visualmated energy us

eens. Each one notifies flashing the nformation sage in the ur goal was green in a

plication is ges users to 5.5° C and analysis of nd reported n Figure 1,

er is setting cates non-w indicates mendation, ommended

dations, the clothing to

vironmental

mmate

space heater

lization of se

Session: Home Heating UbiComp’13, September 8–12, 2013, Zurich, Switzerland

104

impact” of the user’s set temperature point over time by subtracting actual from ideal consumption. Ideal consumption is the energy consumption that would result if the suggested guidelines were followed (power use is estimated using a modified U.S. Department of Energy equation [22], with 5 minute calculation intervals). Although this equation is not tailored to a user’s home, it does react to changes in indoor and outdoor temperature, thus enabling feedback that reflects the particular activity that was changing in this study (the use of the home heating system). We chose this approach because of the prohibitive cost of accurately accounting for details such as the efficiency of the heating/cooling device, the on/off cycles of the thermostat, the volume of home space, the insulating capabilities (R-factor) of the walls, the number of walls exposed to outside conditions, and so on. Instead, our simplified energy calculation sufficed for providing feedback to users and we are also not using it in the quantitative analysis.

Depending on the result, the application either congratulates the user, or provides the environmental impact of their higher setpoint in terms of the number of 60W light bulbs that could have been powered by the excess energy consumed. We use the light bulb metaphor to equate wasted energy to leaving the lights on, a concept that may be easier to understand than the number of Watts wasted.

By clicking on the Recommendation screen, users could move to the Report Screen. As illustrated in Figure 2, the Report Screen offers users a graphical overview of either daily, weekly, or monthly energy use. By including these longer periods of time, the system aims to convey a sense of mounting achievement and provide motivation to continue with and/or improve sustainable behaviors. Each sub-graph includes a goal line highlighting the difference between intended and achieved levels of consumption. Ideal consumption is defined as for the recommendation screen, and the graph uses the same color scheme to indicate compliance with the recommended values. To provide positive reinforcement, a small smiley face is shown if users achieve the setpoint goal for more than half of the days in a particular week or month.

The final screen is the Temperature Control Screen, which displays the current temperature of the house, and a simple interface for changing it.

Study Method We deployed the application to 10 households. We recruited all 10 participants from the same northeastern U.S. city (Table 1), through advertisements on CraigsList and at local universities. Participants were required to have an existing digital thermostat to simplify installation, and T-Mobile or AT&T as mobile phone provider (other U.S. carriers do not support GSM phones). We provided users with an Android-based T-mobile G1 phone and helped them to migrate information from their SIM card and instructed them to use the phone as their own, to carry it

with them at all times, and to remember to keep it charged. We compensated each participant with $20 per week they participated in the study. Our participant requirements did make recruitment difficult. As a result, we allowed people with roommates or families to participate. This affected our data about manual control, since only one household member was given an Android phone. In these cases, we asked questions during interviews about how the participant and other household members shared control over the thermostat. Table 1 provides an overview of participants and their household situations, including commute times. The participants in our study all had commute times lower than 60 minutes, which is in line with the data collected by the U.S. census bureau [21].

Participants used the phone for a total of 5-6 months. During the first 5 weeks we, replaced participant phones with our own, and gathered baseline GPS data to establish home and away zones. The phone collected location data using GPS (sampling every 2 minutes to conserve battery) and sent it to a backend server. To further conserve battery, we only sampled GPS when the user was moving, as determined by the phone’s accelerometer. We also replaced participant digital thermostats with a SmartHome Insteon Thermostat (X10-enabled) that could be remotely controlled via the phone and that we could use to collect indoor temperature. Outside temperature was collected using the Google Weather API. Once the initial set up and baseline location data collection was complete, we enabled the application. At this point, we asked our participants to get in the habit of controlling the thermostat exclusively through the mobile phone application.

We divided the main study into a Baseline phase of 6 weeks designed to help overcome the novelty effect and a Deployment phase lasting an average of 90 days from January 2011 through March 2011. During the Deployment phase, we collected indoor and outdoor temperature data, data about thermostat control events. The average outdoor temperature throughout the deployment was 0.3C, with an average daytime temperature of 1.4C, and an average nighttime temperature of -0.4C. Except 4 days at the end of March, heating was necessary throughout the whole study. In addition to this data collection, we interviewed participants halfway through the Deployment phase and at the end. The goal of the interviews was to find out more about how participants used the system and how this fit into their daily practice around home heating and cooling. The interviews were organized around questions about how the system was used, what participants do to stay comfortable, the impact of roommates, and how participants responded to the recommendations made by the system.

Next we present our process for analyzing the data, followed by quantitative data on how and when users adjusted their home temperature, and qualitative data on our users’ comfort and the overall user experience.

Session: Home Heating UbiComp’13, September 8–12, 2013, Zurich, Switzerland

105

Data Preparation For our quantitative analysis, we gathered information about thermostat setpoint, indoor temperature, and user location (using GPS). We removed on average 1.8 days per user from our analysis due to missing phone data. For each 5-minute period of the day, we calculated whether or not participants were at home. Short trips (such as walking the dog) were counted as home time. We calculated the following metrics for data analysis.

Average Home Temperature: The mean thermostat setpoint when the user was at home

Average Away Temperature: The mean thermostat setpoint when the user was away from home

Accuracy: the percentage of minutes where the indoor temperature matched the user’s preferred temperature (if the participant was at home) or the standard away temperature, 15.5C (if the participant was away from home).

UnnecessaryHeating: the percentage of daytime minutes where the user was away from home, but the temperature was above 15.5C. We further separated UnnecessaryHeating as follows: o Forgetting: the user simply forgot to turn the heat

down when they left the house, meaning the temperature remained high until they returned or until they remembered that they forgot

o Bad Guess: the user turned up the temperature from a remote location with the expectation of going home, but did not return home within 80 minutes. 80 minutes was chosen because it allows for the commute time plus a substantial buffer, and is more than the average heat-up time for each home.

o Quick Heat Ahead: the user turned up the temperature from a remote location with the expectation of going home, and did return home within 80 minutes, but after the home had already warmed up to the desired set temperature.

LostComfort: the percentage of daytime minutes at home where the temperature was below the user’s preferred temperature. We further separated LostComfort as follows: o Normal: the house was still heating up when the user

returned home o No Raise: situations where they chose to keep the

temperature low for at least 90 minutes.

Results and Discussion

We calculated some of our measures using data from the Baseline phase to use as comparison points. User accuracy averaged over all users was 72.5%, UnnecessaryHeating occurred for an average of 302 minutes a day (SD=185) and LostComfort was a problem for an average of 73 minutes a day (SD=29).

During the Deployment, phase, we logged an average of 2.3 thermostat control events per user per day. The vast

majority of temperature adjustments were made for regular heating and cooling tasks, with 10% made because of illness and 11% because of a guest.

Participants liked being able to remotely control the thermostat. However, some participants found the system cumbersome in comparison to their existing practices with the thermostat itself (which, unlike the phone, does not have to be found, charged, etc.). Four of the ten participants reported trying to closely follow the recommendation of 15.5C when away, 18C when home. The average temperature when users were at home was 19.4C, while the average temperature when they were away was 17.2C. Users spent an average of 735 minutes (SD=65) at home and 705 minutes (SD=66) away from home per day.

Their accuracy using the system was 75.1%, an increase of 2.6% over the baseline condition. LostComfort was an issue for an average of 67 minutes a day (SD=10). Two-thirds of this was for Normal heat up, and one-third for No Raise, where participants purposely did not turn up the heat even though they were home. UnnecessaryHeating was an issue for an average of 290 minutes a day (SD=70), of which 96% was caused by Forgetting to turn the thermostat down or forgetting and then Correcting the situation.

Note that the temperature was sometimes controlled directly through the thermostat, either by participants or their housemates. This could have led to an overestimation of both kinds of errors. To address this, we recalculated our metrics only including participants who had no roommates, or reported that their roommates did not use the thermostat (P03, P04, P07, P11). The average number of daily interactions per non-roommate participant was 2.6. Accuracy was 78.8%, an increase of 6.3% over the baseline condition, LostComfort increased by 1 minute to 68 minutes/day (SD=12), while UnnecessaryHeating dropped from 290 minutes to 258 a day (SD=58), with a similar distribution of reasons.

Situated Control Participants had a nuanced understanding of how different actions in their home influenced the flow of air and heat within the spaces they used: I would mostly just kick it on to like, have some like hot air blowing through the vents … so that's like the temperature isn’t necessarily increasing so much, but just like having that air blowing to warm up. (P02). Participants were also sensitive to the fact that some rooms did not need to be as warm (such as the bedroom at night). This management of space was reminiscent of the “ongoing configuration and maintenance” that Woodruff et al. observed in their study of green homeowners [24]. However, unlike with Woodruff’s participants, there was less attention paid to external factors (“nature’s changing state and rhythms” [ibid.]).

Management of comfort, not energy In addition, they varied greatly in terms of their preferred tradeoff between comfort and energy savings, and control and energy savings. For example, one person wanted the

Session: Home Heating UbiComp’13, September 8–12, 2013, Zurich, Switzerland

106

temperature only turned up after she got home, to save the maximum energy, while others did not like being cold for any length of time and were unwilling to reduce their home setpoint: “1 or 2 degree difference is enormous in terms of the comfort level, and so between like what we have it at, and what we find most comfortable which is 71, putting it down to like 70 or 69 gets really cold.” P05.

Comfort was an ongoing theme in our data. Participants managed comfort not only through room temperature (and air flow), and additional heating support (six participants reported using a space heater) but also by using blankets, sweaters, slippers and so on. The use of space heaters, a more wasteful kind of high-energy heating system circumvents the type of manual system we were looking to explore. When we probed as to why participants used space heaters, participants reported that they were used to counteract shortcomings in the insulation of certain rooms in their apartments. In addition, some space heaters were used to accommodate roommate disagreements about overall temperature. Thus space heaters were used purposefully to increase personal comfort. Kuijer and de Jong found similar results in their exploration of heating practices in 60 Dutch households. [14]

On the other hand, some with high setpoints chose to drop their temperature and reported noticing a significant difference in their energy bills: “Before I think the last, the bill before this was for [US$] 131, and this bill was [$US] 55… [partly due to outside temperature change], but I would say that part of that is also caused because of the application.” P10. Four participants reported really trying to follow the recommendation of 15.5C when away, 18C when home. For example, P04 felt positively about the goal in part because she grew up with the house at 13C when away. This impact of personal history on energy consumption was also found in [4].

While we do not have a perfect window into user intent, it is clear that even a well-intentioned user, with an eco-feedback display to guide him or her, will forget to turn the heat down at times. While providing remote access to the thermostat and eco-feedback significantly increased users’ accuracy in controlling their heating system, it is also clear that manual control is not a complete solution to the management of energy use for home heating and cooling. What was more surprising was that, at times, users will choose not to turn the heat up when they are home, as we saw with the existence of No Raise time in our analysis.

OCCUPANCY PREDICTION: THERML As past work and our own study shows, neither programmable thermostats nor a remote control solve the problem of reducing energy consumption of temperature control systems. As an alternative, we now turn to automated approaches to thermostat control.

An important prerequisite for automated techniques is accurate information about when people are at home or will be returning home. In addition to providing novel insights

into how users manage both energy savings and comfort, the remote controlled thermostat that we explored provided us with real-world GPS location data and baseline accuracy metrics against which to test our automated techniques.

An early approach to using GPS location to reduce the energy consumption of heating and cooling is to reduce the indoor temperature based on the current projected return time to one’s home and the heat-up time of the space [10]. By decreasing the home temperature to a point that allows re-heating of the space to the preferred temperature within the time a person needs to return home, the GPS-Thermostat system was demonstrated to reduce the indoor away temperature on average by 1.4C.

As an alternative approach is to predict when people will return home (e.g., [10,16,17,20]). The most successful algorithms have used information about when people have historically been at home (gathered using interaction with home systems [17], RFID tags [20], and other inexpensive sensors [16,18]) to estimate when they will be home in the future. This allows a temperature regulation system to adapt as occupancy patterns change over time, and has achieved occupancy predictions with accuracy above 80% [16,20].

Our TherML occupancy prediction algorithm is based partly on a previous technique that has been used to accurately predict the future locations of pedestrians [26] and taxi drivers [25]. We briefly summarize the technique here and refer the reader to [25] for further details. The technique takes as input, contextual factors like time of day, road network features (e.g., type of road) and users’ past sequences of locations. It uses this information to learn a utility function that makes users’ past trajectories efficient destination-directed solutions for a planning task (e.g., selecting a route home), and infers a probability for each possible solution to similar planning tasks.

The system makes predictions about the future location of an individual by treating her recent locations (i.e., trajectory) as an incomplete solution to a planning task with unknown destination, and probabilistically inferring the remaining locations and intended destination that complete the task. To use this system, we provide it with a prior distribution over historical destinations. Here, we use both time and past locations, based on an algorithm developed by Ashbrook and Starner [2], to generate a context-dependent prior, The prior is a probabilistic distribution, representing the likelihood that a user is going to each possible destination, given the current time.

This system is expected to work well when the user is driving in their car [25]. However, it does not handle the case when the user is stationary or walking. For this case, we calculate a different prior distribution. Given historical location data, we split each day of that data into 5-minute blocks of time, and count how often a person was at each potential destination within a 5-minute block. We then normalize the frequency count to account for locations that appear multiple times within the same 5-minute block (e.g.,

Session: Home Heating UbiComp’13, September 8–12, 2013, Zurich, Switzerland

107

due to multiple GPS readings). When queried about a future time, it predicts where the user will be at that time. For example, if we want to predict where the user is one hour from now, we simply look at the 5-minute block corresponding to that time and select the most probable destination from the prior.

Our hybrid algorithm decides if a person is driving or walking at a given moment in time by calculating the time difference and distance to the last recorded location and uses this information to calculate the current speed. If the current speed exceeds 20 m/s (threshold based on [14]) the algorithm sets the user state to driving and uses the driving prediction system to determine where the user is going; otherwise, it sets the user state to walking (or stationary) and uses the second prior distribution to determine where the user is going.

For each prediction step we consider the three destinations with the highest prediction probability and predict that the user is returning home if one of these three destinations is home. We use the 3 most probable destinations because there are many cases where the probability that the destination is home is just slightly lower than another destination(s), so selecting only the top choice results in reduced prediction accuracy.

VALIDATION We used real-world data gathered during our study of the remote controlled thermostat to simulate and evaluate the performance of TherML and to compare to PreHeat and GPS-Thermostat. We used data gathered during the first 4-5 weeks of the study to calculate the prior distribution over common locations for each user. Using the algorithm from [2], our system identified between 6 and 15 common locations, on a per-participant basis.

We used data from the final 90 days of the study for our simulation to ensure that there was a realistic level of variability in the data, both in user behavior and outdoor temperatures. We made use of participant location data (for evaluating the accuracy of the different occupancy prediction algorithms), and preferred temperature when participants were home, outdoor temperature (for evaluating the impact of the algorithms on LostComfort and UnnecessaryHeating and energy usage) and indoor temperature. We also used this information to calculate the average heat-up time (i.e., the time to heat up a home to the preferred temperature); the average across participant homes was 59 minutes (Figure 3). We calculated this for each participant home, by averaging over the time it took for a home to reach the setpoint temperature, from 5 weeks of data. By measuring this over 5 weeks, we reduce the effect of variations caused by external factors such as open doors and windows, and outdoor temperature variations.

DATA ANALYSIS To test the efficacy of TherML, PreHeat, and GPS-Thermostat, we implemented all three algorithms and ran them on our data. First, we calculated the average indoor

temperature that the algorithms would set each house to, when users were not at home, as a measure of UnnecessaryHeating. In order to simplify the indoor temperature determination we ignored the heat-up period and fluctuation due to the thermostat’s hysteresis and assumed a fixed temperature. Then, for the predictive algorithms, TherML and PreHeat, we calculated for each five-minute interval whether each algorithm would have predicted that the user was at home or away. We used a sixty-minute look ahead for this, the average warm-up time based on real-world data from a similar climate and infrastructure [10], and the average warm-up time for our participants’ homes.

We calculated the following measures (analogous to the metrics used in our user study): Accuracy: number of minutes correctly predicted

divided by the total number of minutes in a day. UnnecessaryHeating: the prediction is wrong,

predicting that the user arrives home before she actually does, which results in unnecessary heating.

Correct: the prediction is correct and there is no unnecessary heating or loss of comfort

LostComfort: the prediction is wrong, predicting that the user arrives home after she actually does, which results in greater energy savings, but (possibly) reduced thermal comfort.

Next we provide some details on our simulated implementation of each algorithm.

Algorithm Simulation We now describe the details of the simulation for the prediction systems. As described earlier, TherML predicts where the user will be in an hour (home or away). This prediction is considered correct when our ground truth data for an hour after the prediction matches the prediction. A user was considered to be home if they were within a 40-meter radius of the GPS coordinate for their home. For purposes of comparison, we also implemented the Preheat algorithm [20]. We had to slightly adapt their approach to our setting. First, instead of using RFID tags to determine when users arrived or left home, we used the smartphone GPS, as in TherML. Second, Preheat originally used 15-minute intervals to make predictions, but we change that to 5-minute intervals to match our system. To ensure equal

Figure 3: Time (in minutes) needed to heat each participant’s home

0

10

20

30

40

50

60

70

80

P02 P03 P04 P05 P06 P07 P08 P09 P10 P11

Time in

mins

Session: Home Heating UbiComp’13, September 8–12, 2013, Zurich, Switzerland

108

sf

ATtiloathtodO1hwi

ImWthtpwsccPininanp

Tttitisathhttitaodctho

FtBtthmTtha

settings for botfor PreHeat as w

As a baselineThermostat algime it would taocation. Using

average changehe set indoor to rise to the p

decided to leavOur participant1), compared theating systemwhen no one iss limited for ou

mpact on IndoWe now calculhe indoor tememperature is

provides an indwhen no one spent energy ican also be usecompare the inPreHeat to GPSndoor temperncorrectly pre

and we used a no one is hompredicted as aw

The GPS-Thermemperature seime to return hime it takes to

soon as an occua temperature the preferred te

home immediaemperature is ime it takes takes to change

our simulatiodetermined the collected indoohe collected

outputs of the a

Figure 4 showemperature ac

Both TherML emperature clheir prediction

minimizing UnThermostat alghe inside temp

across all user

th algorithms, wwe used for ou

e case, we agorithm [10]. Gake for the useg the average e in degrees Ctemperature sopreferred indoove for home frts that had sucto the heat-up

m must maintas home. Althouur data, it prov

oor Temperatulate the impact

mperature whens an analog dication of eneis home. Calc

is difficult, bued for comparinndoor temperaS-Thermostat, rature for cadicted to be arecommended

me (15.5°C) [5way.

mostat algorithtpoint, which

home from the o change the inupant leaves hthat would all

emperature if thately from the

determined bo return homee the temperaton of the

heat-up time por temperature

GPS data talgorithm.

ws the averagcross all user

and PreHeat ose to 15.5Cn accuracies nnecessaryHeagorithm is onlyperature by an

rs. Due to the

we used the saur approach.

also implemeGPS-Thermoster to get home heat-up time

Celsius per mino that the tempor temperaturefrom the currench short commutimes of their

ain a high indugh the value

vides a useful b

ure t of the differenn the user is n

to Unnecessargy spent on hculating the e

ut average indng the differenature impact owe used each

ases when that home (Unned indoor tempe], for cases th

hm generates uses the currecurrent locatio

ndoor temperahome the setpolow re-heating he user would current locatioby dividing the by the time ture by 1C. F

algorithm wper user from the data and uso simulate th

ge away indos by algorithmhave an indo

C, implying thare quite hig

ating. The GPy able to redun average 1.4

relatively sho

ame training da

nted the GPat calculates thfrom her curreit calculates thnute and adjus

perature has time if the occupant location [10ute times (Tabhomes, that th

door temperatuof this approa

baseline.

nt algorithms oot home. IndoaryHeating, anheating the houexact amount door temperatunt algorithms. Tof TherML anuser’s preferr

he users weecessaryHeatinerature for whhat are correct

a sliding indoent location anon, as well as thature by 1C. Aint is reduced of the space decide to retu

on. The reduche it

For we he ed he

oor m.

oor hat gh, S-

uce C ort Fi

ata

S-he

ent he sts me ant 0]. ble he

ure ch

on oor nd

use of

ure To nd ed

ere ng) en tly

oor nd he As to to

urn ed

distancnot havmore.

AccuraSince particippredicteach uaccurac90.6%controlTherMall useboth al

In ordealgorithalgorithwhen tbefore phone readilymodern

igure 5 Average

Figure

ces that our pave the opportu

acy GPS-Thermo

pants, we focutive algorithm

user, is shown cy of 92.1% an. Both signifil condition w

ML outperformeers by an averalgorithms were

er to better unhm and PreHhm should be uthe two algoritit is possible and derive the

y available GPSn feature and s

e Accuracy by U

e 4: Average IndUser Whil

articipants movunity to reduce

ostat performus our remain

ms. Accuracy in Figure 5.

nd PreHeat hacantly outperfwe deployed ed PreHeat foage of 1.5%. e statistically si

nderstand the Heat, especiallused, we also thms diverge into implement

e necessary occS location systsmart phones.

User for TherM

door Temperatle The User Is A

ved, GPS-There the indoor te

ms poorly ning analysis ofor each algoTherML had ad an average aform the manu

(accuracy oor every user, The differenceignificant (p<0

differences bely to show winvestigated mn accuracy. AsPreHeat using

cupancy historytem implement

ML and PreHea

ure For Each AAway From Hom

rmostat did emperature

with our on the two orithm, for an average accuracy of ual remote

of 78.8%). and across es between 0.001).

etween our when each

more deeply s we stated g a mobile y using the ted in most

at

Algorithm, Per me

Session: Home Heating UbiComp’13, September 8–12, 2013, Zurich, Switzerland

109

For that reason, our implementation of both algorithms have comparable costs.

Our algorithm uses the occupant’s current GPS location to predict the path and future location of a moving target. To better understand the accuracy differences between our algorithm and PreHeat, we calculated the amount of time a user was away from home and from this, the percentage of away time that the user was driving. Our users on average drove 13.3% (SD=3.2%) of the time they were considered away from home. We calculated the prediction accuracy as a function of distance for both algorithms. Figure 6 shows the difference in prediction accuracy by distance driven between TherML and PreHeat. We can see that the accuracy difference is correlated to the length of the drive: the greater the length of the drive the greater the accuracy gain of TherML over PreHeat. This means that TherML is more effective for people with longer (but still reasonably short, < 14 mile) drives.

Impact on Thermostat Control and User We now present an analysis of the different prediction algorithms for controlling the thermostat. Here we considered 3 metrics: UnnecessaryHeating, LostComfort and Correct. We calculated these by summing performance across each five-minute interval, and averaging across all days and users (see Figure 7 and Table 2). The differences across the algorithms for each metric are statistically significant (p<0.001). The Correct times for each algorithm mirror the prediction accuracies reported earlier.

The difference in accuracy between TherML and PreHeat results in an improvement (decrease) in UnnecessaryHeating (13.2 minutes) and LostComfort (9 minutes), averaged across all users and days of the study.

DISCUSSION Our approach, TherML, for predicting occupancy was very accurate – 92.1%. It was significantly better than PreHeat (by 1.5%), than our manual remote control deployment (by 13.3%), and than our manual baseline (by 19.6%). While the overall average accuracy improvement over PreHeat is small, we showed that this improvement increases, by up to 7%, when users spend more time in their car, driving longer trips. Note that the longest drives in our study were

reasonably short (14km); we expect even greater improvements over PreHeat with even longer drives.

We also introduced three metrics to better understand the impact of the increased accuracy: UnnecessaryHeating, LostComfort and Correct. By improving on accuracy, TherML showed an improvement in all of these metrics, in comparison to PreHeat. However, despite these improvements, we do not expect TherML to result in a decrease in overall energy usage. It would decrease energy consumption as a result of decreased UnnecessaryHeating, but would increase energy consumption as a result of decreased LostComfort. Decreasing LostComfort and therefore increasing Correct, means that a user’s home is heated more when the user is home when using TherML. Despite this apparent contradiction, TherML performed better in both reducing UnnecessaryHeating and LostComfort, simultaneously optimizing energy usage and user comfort. We discuss the tradeoffs between the different algorithms used in our analysis, and then address this result which questions whether improving accuracy of occupancy prediction is the answer to reducing energy consumption due to heating a home.

From an implementation standpoint, the three algorithms we simulated have roughly the same costs. GPS-Thermostat requires a mobile phone with GPS and Internet access to contact a computer to determine the current driving time to return home and to control the thermostat. Similarly, our implementation of PreHeat and TherML have the same requirements; both need access to a computer to predict when the user will be home, although all 3 algorithms could be made to run directly on the phone. However, the original implementation of PreHeat made a different tradeoff, using RFID tags to determine home occupancy rather than a mobile phone. The RFID tags require instrumentation of personal artifacts (e.g., keys) that the user always has with

Figure 6. Accuracy Difference between TherML and PreHeat as Function of Distance Traveled. Note: x-axis is not uniform.

Table 2. Correct, UnnecessaryHeating and LostComfort Average (and Std. Dev.) in Hours Per Day, for Each Algorithm.

Correct Unnecessary Heating

Lost Comfort

TherML 22.11 (0.14) 0.94 (0.11) 0.95 (0.09)

PreHeat 21.74 (0.18) 1.16 (0.15) 1.10 (0.10)

Figure 7. UnnecessaryHeating, Correct, and LostComfort Average Per Day, for Each Algorithm

Session: Home Heating UbiComp’13, September 8–12, 2013, Zurich, Switzerland

110

her when she leaves her home, but comes without the cost of making sure she always has a charged phone. The phone is being used for other activities than managing home heating, so there is external incentive to keep it with you and charged. Unfortunately if the phone is running out of battery the thermostat will not be controlled any more as the control is being realized through the phone. TherML’s computational costs are slightly higher as its prediction algorithm is more sophisticated.

While TherML worked best for our participants’ data, it may not be optimal for all settings. We now discuss the situations in which each of the automated approaches should be used. PreHeat had almost the same performance as TherML for our participants. For users that would prefer an RFID-based system over a smartphone system (e.g., privacy, prefer to give key but not phone to children), the original implementation of PreHeat could be used at the cost of a loss in both energy savings and comfort. However, our implementation of PreHeat that uses the same technology as TherML should not be used, as the loss in energy savings and comfort is not balanced with any gain.

GPS-Thermostat performed the worst among the automated techniques for our participants. However, it is clear that it could perform better if users’ occupancy of their home is not predictable. This could occur in the case of shared student housing, for example. GPS-Thermostat could also be preferred if users’ commutes are really long and users stay far away from their home (e.g., not running errands close to their home), allowing the home to reach its away temperature for significant periods of time. However since the mean commute time for most Americans is well less than an hour [21], the approach still has limitations in applicability. One last issue to note is that GPS-Thermostat has no LostComfort, trading it off for increased UnnecessaryHeating. For users that prefer comfort over reduced energy use/costs, this approach may be appropriate.

TherML performed the best for our participants, in accuracy and in optimizing UnnecessaryHeating and LostComfort. It should be used over PreHeat when the use of phones is not a concern, and definitely when users are driving medium distances (including commutes and other drives). It remains as future work to identify the driving distance or time threshold at which TherML should be swapped out for GPS-Thermostat.

Despite the superiority of the automated prediction systems in terms of accuracy over our manual remote system, we did learn valuable lessons from our initial deployment. We now explore how these automated techniques can be improved with insights from our deployment of an eco-feedback system with a remote control. We found that users were uniformly less accurate in controlling their thermostat, than either TherML or PreHeat. This is not a particularly surprising result, as part of the motivation for an automated system was that users were unable to accurately program their thermostats or maintain efficiency in heating their

home. However, when we explored data further, we found two particularly interesting results. First, LostComfort was divided into 2 parts: home warming up after the user was already home (67%), and the user being home and choosing not to raise the temperature (33%). Both of these indicate opportunities to be “unoptimal” in heating a home. Whether users intentionally wait to heat their home even though they may experience some discomfort, or whether this is unplanned, the fact remains that they often come home to a home where the temperature is not at their desired setpoint. One-third of the LostComfort time, the user is explicitly not raising the temperature when they are at home. This calls for further exploration of systems that may penalize errors that lead to UnnecessaryHeating more than errors that lead to LostComfort to obtain greater energy savings.

Second, for UnnecessaryHeating, almost all of it was caused by users leaving the house and not turning the thermostat down. One-quarter of this UnnecessaryHeating time occurred when a user did this, but later remembered to reduce the temperature and did so remotely. But three-quarters of the time, the temperature remained high the entire time the user was out of the home. There are a number of possible implications for a mixed-initiative thermostat control system. Such a system could let the user remain in control and just remind the user when they are leaving the house to lower the temperature. An alternative worth exploring is to let the user control the temperature when returning home, since from our LostComfort analysis there are energy savings to be had, and automatically control the temperature when the user leaves the house.

Further opportunities for customizing heating plans to users’ routines and preferences will be explored in future work. First, we could explore the maximum heat up time that participants are willing to experience, and start heating the home less than an hour before they return home. Another option is to use activity recognition to understand when a user may be exerting herself, providing an opportunity to reduce the temperature. As well, we would like to explore how predicting when a user is leaving the house can impact energy consumption. Perhaps energy consumption can be reduced by lowering the temperature before a user is predicted to leave. As with any energy consumption intervention, they will have to be deployed and evaluated in realistic settings. In the future, we will continue to optimize TherML for not just occupancy prediction, but energy reduction and user comfort.

CONCLUSION In this paper, we presented TherML, an occupancy prediction algorithm that has very high accuracy based simply on data from a GPS-enabled phone. Our first contribution analyzes the deployment of an eco-feedback system that allows users to remotely control their thermostat. We found that with our system, users improved their accuracy of controlling their heating system by as much as 6.3% and reduced wasted energy due to UnnecessaryHeating by 44 minutes/day. We also found that

Session: Home Heating UbiComp’13, September 8–12, 2013, Zurich, Switzerland

111

users, at times, were willing to live with decreased comfort when they are at home, and that they consumed large amounts of energy by not reducing the set temperature of their thermostat when they left home. Our second contribution is the first comparison of competing occupancy prediction techniques, which shows that TherML is significantly (if slightly) more accurate than PreHeat, and discovered that when the user is driving, TherML performs much better than PreHeat. This comparison also shows that TherML heats the house much less when nobody is home, than GPS-Thermostat for our dataset. We examined the tradeoffs for each approach and identify settings for when the different approaches could be used. We found that despite the accuracy increase of TherML, there was minimal overall energy savings to be had, due to offsets in UnnecessaryHeating and LostComfort. Increased accuracy about home return time was one reason for the minimal savings, however data from our deployment suggests that this may not match what users optimize for. By combining these results, we suggest opportunities for both an improved automated system and a mixed-initiative system for controlling one’s thermostat.

ACKNOWLEDGEMENTS This work was supported in part by NSF grants IIS-1017429, 074628, 1217929, and 0916459, Google, and the Portuguese Foundation for Science and Technology grant CMU-PT/HuMach/0004/2008 (SINAIS).

REFERENCES 1. American Society of Heating, Refrigerating and Air-

Conditioning Engineers (ASHRAE). Energy Standard for Buildings Except Low-Rise Residential Buildings. ASHRAE Standard 90.1-2010, 2010.

2. Ashbrook, D. and Starner, T. Using GPS to learn significant locations and predict movement across multiple users. Personal and Ubiquitous Computing, 7(5): 275–286, 2003.

3. Chappels H., Shove E. Debating the future of comfort: Environmental sustainability, energy consumption and the indoor environment. Building Research & Information 33(1), 31-40, 2005.

4. Dillahunt, T., Mankoff, J., Paulos, E. and Fussell, S. It’s not all about green: Energy use in low-income communities. Proc. Ubicomp 2009, 255-264.

5. Dong B., Integrated building heating, cooling and ventilation control, Ph.D. Thesis, Carnegie Mellon University, 2010.

6. Environmental Protection Agency. Summary of research findings from the programmable thermostat market. Office of Headquarters, 2004.

7. Karjalainen S. Thermal comfort and user of thermostats in Finnish homes and offices, Building and Environment 44, 1237-1245, 2009.

8. Fischer C. Feedback on household electricity consumption: a tool for saving energy? Energy Efficiency 1, 2008.

9. Froehlich, J., Findlater, L. and Landay, J. The design of eco-feedback technology. Proc. CHI 2010, 1999-2008.

10. Gupta M., Intille S. S., and Larson K., Adding GPS-control to traditional thermostats: An exploration of potential energy savings and design challenges. Proc. Pervasive 2009, 95-114.

11. He H. A., Greenberg S., and Huang E.M. One size does not fit all: Applying the trans-theoretical model to energy feedback technology design. Proc. CHI 2010, 927-936.

12. Kaasinen E,. User needs for location-aware mobile services. Personal and Ubiquitous Computing 7(1): 70-79, 2003.

13. Karjalainen S. and Vastamäki R. Occupants have a false idea of comfortable summer season temperatures. In: Seppänen O , Säteri J eds. Proc. of Climate 2007 WellBeing Indoors. Helsinki: FINVAC - The Finnish Association of HVAC Societies, 2007.

14. Knoblauch L. R., Pietrucha M. T., and Nitzburg M., Field studies of pedestrian walking speed and start-up time. Transportation Research Record No.1538, Pedestrian and Bicycle Research, 1996.

15. Kuijer L., de Jong A. Exploring practices of thermal comfort for sustainable design. Workshop on Everyday Practice and Sustainable Design, CHI 2011.

16. Lu J., Sookoor T., Srinivasan V., Ge G., Holben B., Stankovic J., Field E., Whitehouse K. The Smart Thermostat: Using occupancy sensors to save energy in homes. Proc. ACM Conference on Embedded Networked Sensing Systems (SenSys 2010), 211-224.

17. Mozer M. C. The Neural Network House: An environment that adapts to its inhabitants. In M. Coen (Ed.), Proc. of the American Association for Artificial Intelligence Spring Symposium on Intelligent Environments, 110-114.

18. Nest Labs. http://www.nest.com 19. Peffer T., Pritoni M., Meier A., Aragon C., Perry D., How

people use thermostats in homes: A review. Building and Environment 46, 2529-2541, 2011.

20. Scott J., Brush A.J., Krumm J., Meyers B., Hazas M., Hodges S., and Villar N. PreHeat: Controlling home heating using occupancy prediction. Proc. UbiComp 2011, 281-290.

21. U.S. Census Bureau Commuting in the United States (http://www.census.gov/prod/2011pubs/acs-15.pdf), 2009

22. U.S. Department of Energy Central air conditioning life cycle cost alculator, http://www.energystar.gov/ia/ business/bulk_purchasing/bpsavings_calc/Calc_CAC.xls

23. U.S. Energy Information Administration (eia): Residential Energy Consumption Survey (RECS): End-use consumption. 2009. (http://www.eia.gov/consumption /residential/data/2009).

24. Woodruff, A., Hasbrouck, J. & Augustin, S. A bright green perspective on sustainable choices. Proc. CHI 2008, 313-322.

25. Ziebart. B., Maas, A., Dey, A.K., and Bagnell, J.A. Navigate Like a Cabbie: Probabilistic Reasoning from Observed Context-Aware Behavior. Proc. Ubicomp 2008, 322-331.

26. Ziebart B. D., Ratliff N., Gallagher G., Mertz C., Peterson K., Bagnell J. A., Herbert M., Dey A. K., and Srinivasa S. Planning-based prediction for pedestrians. Proc. IROS 2009.

Session: Home Heating UbiComp’13, September 8–12, 2013, Zurich, Switzerland

112