T h e U s a b i l i t y o f a 2 0 1 3 A 6 A u d i I n -V e h i c l e I n f o t a i n m e n t S y s t e m

Prepared for: University Audi Prepared by: Sheena Kapur and Emma Bulajewski Date: December 23, 2013

2

Table of Contents

Table of Contents ................................................................................................................ 2

Executive Summary ........................................................................................................... 4

Background ........................................................................................................................ 5

Research Goals ................................................................................................................... 6

Methodology ...................................................................................................................... 6

Participants .................................................................................................................... 6

The Setup ....................................................................................................................... 7

The Tutorial and Tasks ................................................................................................. 7

The Tutorial ............................................................................................................... 7

The Tasks ................................................................................................................... 8

The Data Collection Methods........................................................................................ 9

Post-task questionnaires .......................................................................................... 9

Post-test questionnaire ............................................................................................. 9

Participant comments ............................................................................................. 10

The Results ....................................................................................................................... 10

Task 1 Bluetooth sync ................................................................................................. 11

Task 2: Voice Commands ............................................................................................ 11

Task 2a: Getting directions to an address: ............................................................ 12

Task 2b: Getting directions to a location ............................................................... 12

Task 2c: Calling ........................................................................................................ 12

Task 2d: Radio ......................................................................................................... 13

Task 3: Temperature Controls ................................................................................... 13

Task 3a: Syncing passenger and driver temperatures ......................................... 13

Task 3b: Syncing all four zones .............................................................................. 14

Task 4: Touchpad ........................................................................................................ 14

Task 4a: Finding a contact in phone book ............................................................. 14

Task 4b: Scroll around the map ............................................................................. 14

Overall Experience Data ............................................................................................. 15

Conclusion and discussion .............................................................................................. 16

Conclusion .................................................................................................................... 16

Discussion ..................................................................................................................... 17

3

Appendix 1: Screener Survey ......................................................................................... 18

Appendix 2: Consent form .............................................................................................. 25

Appendix 3: Participant Questionnaire ......................................................................... 27

Task Evaluation ........................................................................................................... 27

Syncing the phone ................................................................................................... 27

Voice Commands ..................................................................................................... 29

Infotainment Controls ............................................................................................. 31

Touchpad ................................................................................................................. 33

Post-Test Questionnaire ............................................................................................. 35

Appendix 4: Facilitator script ......................................................................................... 37

Facilitator Greeting ..................................................................................................... 37

Tutorial ......................................................................................................................... 38

Tasks ............................................................................................................................. 38

Thank and Wrap-Up .................................................................................................... 39

Appendix 5: Survey data ................................................................................................. 41

Appendix 6: Analyzed qualitative data .......................................................................... 41

Appendix 6: Raw notes ................................................................................................... 41

Participant 1 ................................................................................................................ 41

Participant 2 ................................................................................................................ 44

Participant 3 ................................................................................................................ 46

Participant 4 ................................................................................................................ 49

Participant 5 ................................................................................................................ 51

Participant 6 ................................................................................................................ 53

Participant 8 ................................................................................................................ 56

Participant 9 ................................................................................................................ 59

Participant 10 .............................................................................................................. 62

Works Cited ...................................................................................................................... 66

4

Executive Summary The purpose of this study is to evaluate the usability of the 2013 Audi A6 in-vehicle infotainment (IVI) system. It is important to make IVI systems usable to increase profits, the Audi brand reputation, and safety on the road. The following is a brief summary of the study results: Overall experience:

1. The Audi IVI system received a failing system usability scale score and a negative net promoter score. However, when participants reflected upon their emotions while using the system with the emotion wheel there was a high response rate for strong feelings of interest and amusement. This shows that this system is intriguing and fun for users, but that the system needs to be more usable to maintain and enhance the Audi brand reputation and increase sales of Audi vehicles and their IVI systems.

Bluetooth: 2. The majority of participants, while likely to use Bluetooth, were confused by

the prompts for connecting their phone to the car, causing low appeal for the connection process. Participants desired more “step-by-step” instructions and more feedback from the system if their device was unable to connect to the car.

Voice commands: 3. Voice commands had low experience, appeal, and likely usage ratings across

the board. Participants want to use more natural voice commands with the system.

Temperature buttons: 4. Overall, participants had no complaints about the temperature buttons. The

only negative feedback from participants was that they felt they did not know there were multiple temperature zones in the vehicle.

The touchpad: 5. Participants thought the touchpad was interesting but desired the touchpad

to have more gesture recognition capabilities. Participants were also confused about the touchpad’s modality changes (i.e. why the touchpad’s functionality automatically changed when the user was in their telephone directory but not when the user was looking at the map on the multimedia interface).

Screen expectations: 6. The majority of participants expected the multimedia interface screen to be a

touch screen.

5

Background Our study addresses questions focused around efficiency, effectiveness, and satisfaction with regards to a 2013 Audi A6 IVI system. The reasons for these foci are two-fold: to increases sales revenue and safety for drivers. In 2011, 3,331 people were killed in the United States in crashes involving a distracted driver, compared to 3,267 in 2010i. Furthermore, an additional 387,000 people were injured in the United States in motor vehicle crashes involving a distracted driver, compared to 416,000 injured in 2010ii. It is reasonable to assume that an IVI system that limits the number of manual, visual, and cognitive distractions a driver has to deal with will limit the number of injuries and deaths that occur per year due to distracted driving. While we can assume this is true, users also need to want to interact with the IVI system or else the benefit of the safer IVI system will never be realized. In addition to the increased on-road safety that will come from usable IVI systems, automotive infotainment is set to grow due to consumer’s desires for a more seamless interaction with the outside world while inside their cars. Automotive infotainment revenue has grown in 2011 to 32.5 billion dollars, which is 3.4 percent increase from 2010iii. This growth was in addition to a 4.6 percent increase in revenue in 2010iv. Following the expansion of the automotive infotainment market in 2010 and 2011, the global automotive infotainment market reached $34.6 billion dollars in 2012v. Growth is projected to be even stronger in the next four years, with a revenue increase ranging from 4.7 to 6.4 percentvi. By 2016, the market is projected to reach $41.2 billionvii. This increasing market, coupled with the fact that the automotive infotainment market has been primarily dominated by the aftermarket sector, provides an opportunity for original equipment manufacturers, such as Audi, to enter the market, gain significant profits, and become industry leaders with regards to the in-vehicle experience.

6

Research Goals As researchers, we are trying to determine:

1. a. What facets of the Audi IVI system cause participants trouble?

a.i. What parts of the system do not allow users to effectively complete their tasks?

a.ii. What parts of the system do not allow users to efficiently complete their tasks?

b. What facets of the Audi IVI system do participants like? b.i. What parts of the system allow users to effectively complete

their tasks? b.ii. What parts of the system allow users to efficiently complete

their tasks? 2.

a. Why do certain facets of the Audi IVI system cause participant’s trouble?

a.i. Why does the system not allow users to effectively complete their tasks?

a.ii. Why does the system not allow users to efficiently complete their tasks?

b. Why do participants like specific facets of the Audi IVI system? b.i. Why do these aspects of the system allow users to effectively

complete their tasks? b.ii. Why do these aspects of the system allow users to efficiently

complete their tasks? 3.

a. Are the facets that cause users problems dissatisfying to users? If so, why?

b. Are the facets users like satisfying to them? If so, why?

Methodology

Participants Our nine participants are all from the Seattle-metro area and are all between the ages of 25 and 55, own a car (no trucks or vans), are not students or unemployed, have a valid driver’s license, drive over 2-3 times a week, enjoy driving, own a Smartphone, and make over $40,000 dollars a year. These criteria highlight early adopters of IVI technology; if a system is not usable for early adopters, the product will fail. We confirmed that our participants met the inclusion criteria through a screener survey, located in Appendix 1. As gratuity for participating in our study, we gave participants a stainless steel Audi water bottle, valued at approximately twenty

7

dollars.

The Setup We conducted this study in a 2013 Audi A6 that was parked in a parking lot located at 4555 Roosevelt Way NE, Seattle, WA 98105 (Figure 1). We had a video camera set up in the back seat in, along with a note taker (Figure 2). The facilitator sat in the passenger seat and the participant sat in the driver’s seat (Figures 3, 4).

Figure 1: 2013 Audi A6 in the parking lot Figure 2: Note taker and camera in the back seat

Figure 3: Facilitator in the passenger seat Figure 4: Participant in the driver’s seat

The Tutorial and Tasks

The Tutorial Before we began the tasks, we went over a brief tutorial of the Audi IVI system with the participants. We showed user’s multimedia interface (MMI) home screen and how to access on-screen categories with the main rotary dial. We also showed participants the location of the settings button for each category. We then defined the term voice command, showed participants the location of the voice command button, and how to know when the system is listening for their command (i.e. the

8

“beep”). Finally, we showed participants the location of the touchpad and explained how it can be used as an alternate to some buttons. We gave them the example of how the touchpad can be used to find someone in their phonebook rather than using the main rotary dial, but did not show them how to do this since it was a task. To view the entire tutorial, please refer to Appendix 4.

The Tasks 1. Sync your phone to your car for Bluetooth, hands-free, calling

capabilities This task is important because wireless connectivity within cars is very popular and is projected to grow. In 2012 there was a 35% attach rate for Bluetooth connectivity in vehiclesviii. Furthermore, global original equipment manufacturer (OEM) market revenue for wireless technology in cars is projected to increase by 41 percent, from $1.1 billion in 2012 to $1.6 billion in 2018ix. The projected growth of head-unit connectivity gives Audi the chance to create the best experience for users to connect their devices to their vehicle, which will further separate the Audi brand from the rest of the top tier car manufacturers. Also, this feature is critical for limiting manual distractions and therefore must be easy to set up so users actually use the feature. This task will show us if and why the Audi IVI Bluetooth connectivity process with a cell phone is effective and efficient.

2. Use voice commands to control the in-dash navigation system a. Please get route guidance to 5038 17th Ave NE, Seattle WA 98105. b. Call a specific person in your phonebook. c. Please get route guidance to “Pagliacci Pizza”. d. Change Sirius radio to a different Sirius radio channel.

The safest way to control the in-dash navigation system is via voice command since voice commands allow the driver to keep his/her hands on the wheel and eyes on the road. Furthermore, the increase in voice-recognition units in vehicles is projected to drive revenue from 81 million dollars gained in 2011 to 170 million dollars in 2019x. Additionally, voice commands are now present in many electronic devices and consumers expect to have a consistent interaction paradigm with all of their devices, including in their cars. Voice recognition is a growing market that Audi can dominate to separate itself from other top car manufacturers while also promoting safe driving habits.

3. Using the temperature controls: a. Input the driver side climate control settings to a pleasant 72 degrees

F. b. Sync the driver side temperature control to the passenger

temperature control setting. c. Sync all 4 temperature zones of the car.

9

Setting the interior temperature of the vehicle should be a simple task that does not distract the driver. It should be effective and efficient. This task will allow us to determine whether these controls are intuitive and to assess whether or not icon designs are clear. Having participants perform this task will give us insight if the interface is designed to limit distractions.

4. Use the touchpad to: a. Find a person in your phonebook. b. Change a radio preset. c. Scroll/pan the map on the navigation screen.

This task will allow us to assess how easily the participant can complete specific tasks with the touchpad. We can determine whether using this technology is effective, efficient, and satisfying to users since it is a novel technology in a car.

The Data Collection Methods The data we received came from post-task questionnaires, a post-test questionnaire, and verbal comments made by participants.

Post-task questionnaires We used self-administered post-task questionnaires so the experience, right after each task, was still fresh in participant’s minds. The post-task survey questions were centered on overall experience, appeal of the feature, meaning the extent to which participant’s would desire the feature, and likely usage of the feature to determine what facets of the Audi IVI system cause participant’s trouble and what facets participants liked. Furthermore, we also utilized open-ended questions to gain insight as to why participants liked certain features and why others caused them problems. To view the post-task questionnaires, please refer to Appendix 3.

Post-test questionnaire We used a self-administered post-test questionnaire to assess participant’s overall experience with the Audi IVI system. We utilized a recommendation scale to attain a net promoter score (NPS), showing us user’s loyalty to the Audi brand. We implemented Plutchik’s emotion wheel since emotions can be felt at varying intensities and can mix with each other to form different emotions. This wheel helped us understand the full range of emotions felt by participants while using the IVI system. Finally, we used the system usability scale (SUS) to evaluate the entire system since the SUS has been proven to be reliable and validxi. The SUS has been shown to be more reliable and can detect differences via smaller sample sizes compared to homemade questionnaires and other commercially available scalesxii. Furthermore, the SUS has also been shown to successfully differentiate between unusable and usable systems better than, or as well as, other questionnairesxiii. The complete post-task questionnaire is located in Appendix 3.

10

Participant comments We also recorded and reviewed participant comments as a supplement to the questionnaire data. While the questionnaire data told us what facets of the Audi IVI system are troublesome and likeable for participants, their comments illuminated why certain aspects of the IVI system are troublesome and likeable, and if those aspects are satisfying or dissatisfying to them.

The Results Bluetooth: Severity: Medium The majority of participants, while likely to use Bluetooth, were confused by the prompts for connecting their phone to the car, causing low appeal for the connection process. The current, and increasing, popularity of wireless connectivity is the reason for the severity rating of medium. Voice commands: Severity: High Due to the low experience, appeal, and likely usage ratings for the voice command tasks, coupled with the fact that revenue from in-vehicle voice recognition units is projected to increase more than one hundred percent in 2019, it is imperative to make this feature user friendlyxiv. A user-friendly voice recognition system will allow Audi users to maintain a consistent interaction paradigm with all of their devices while keeping users safe. Audi has a chance to not only be present in this market, but also to create the best user experience for voice recognition features and dominate this market. Temperature buttons: Severity: Low Overall, participants had no complaints about the temperature buttons. The only slightly negative comment was that participants didn’t know there were multiple temperature zones. The touchpad: Severity: Medium Participants thought the touchpad was interesting but desired the touchpad to have more gesture recognition capabilities. Participants were also confused about the touchpad’s modality changes (i.e. why the touchpad’s functionality automatically changed when the user was in their telephone directory but not when the user was looking at the map on the MMI). However, participants were very excited about the touchpad technology in the car. If Audi can improve the technology to better fit the user’s mental model, Audi will have a distinct advantage over other car manufacturers.

11

Screen expectations: Severity: Low The majority of participants expected the MMI screen to be a touch screen. However, most participants were still able to complete all tasks. Overall experience: The Audi IVI system received a failing SUS score and a negative NPS. However, when participants reflected upon their emotions while using the system via the emotion wheel, there was a high response rate for strong feelings of interest and amusement, showing that this system is intriguing and fun for users, but that the system needs to become more usable to maintain and enhance Audi’s brand reputation, and increase sales of Audi vehicles.

Task 1 Bluetooth sync Although 7/9 participants were able to sync their phone, these participants experienced significant confusion while doing so. Participants expressed their confusion numerous times while attempting to sync their phones making statements such as, “I’m a bit confused as to why it wouldn’t pair up”, “Is it going to work?”, and “Not sure what to do now...”. One participant felt the success of syncing his phone was “not promising” due to his confusion on how to proceed, evident by him saying, “maybe I need to be in some setting on my phone. Maybe I need pairing mode? Not sure. Now it’s searching. Hands free? Don’t know what that means.” To view all of the organized qualitative data, please refer to Appendix 6. Task: Syncing the phone to the car for “hands free” calling capabilities

Overall experience rating 5.6 out of 10 (+- 3.0)

Appeal rating based on usage during study

3.7 out of 5 (+- .9)

Likelihood of use 4.6 out of 5 (+- 1)

Design Recommendation: To minimize confusion, provide step‐by‐step prompts for Bluetooth syncing, including what features need to be activated on the phone, when the features need to be activated, and where to find these features (i.e. enable Bluetooth via the settings screen on the device and make sure pairing mode is on). Additionally if the phone is unable to sync, provide helpful feedback for users.

Task 2: Voice Commands 8/9 participants did not initially realize that the system only understands commands with specific keywords (e.g., “Navigate”) at the beginning of spoken sentences. Overall, participants had difficulty controlling the IVI system with only

12

voice commands. Specifically, participants desired the ability to speak naturally to the system. To view all of the organized qualitative data for each sub-task, please refer to Appendix 6. Task: Using voice commands Overall experience rating 3.3 out of 10 (+- 1.6)

Appeal rating based on usage during study

2.4 out of 5 (+- .7)

Likelihood of use 2.8 out of 5 (+- 1.1)

Task 2a: Getting directions to an address: While performing this task, a majority of participants did not use the “enter address” command the first time. Participants used other commands they expected the system to understand such as, “get directions to...”, “find directions to...”, “locate...” Design Recommendation: Have the system understand a variety of commands, not just one. For example, have the system understand keywords such as, “navigate”, “find directions”, “directions”, and “locate”.

Task 2b: Getting directions to a location More than half of the participants did not use the “Enter online destination” voice command the first time they tried getting directions to a specific location. One participant commented, “I don’t even understand what ‘online destination’ means” Design Recommendation: Have the system understand a variety of commands, not just one. For example, have the system recognize a search term by merely saying the location’s name and “find location”.

Task 2c: Calling The majority of participants expected that saying the first name of a contact they wished to call was sufficient to begin the call or search for a contact, rather than saying the first and last name. One participant remarked, “eventually I figured out it’s very sensitive to last names. It probably wants to know the last name. I kind of expected it would pick the only Kristina in my phone”. Another participant commented that she “shouldn’t have to say the full name.” Design Recommendation: The system should be able to search for a contact/place the call using first names only. If more than one contact with the same first name exists, then prompt the user to then say the last name of the contact he/she wishes to call.

13

Task 2d: Radio Most participants expected that saying a station name or the station number would indicate to the system to switch to a different radio station. One participant commented that he was “not sure how to set or get to a station I actually want. I want to be able to say ‘Sirius station 094’”, while another participant “expected I could say the name of the radio station or the number”. Another interesting observation is that a few participants tried to use the “next page” command, which is available on the help screen, with the radio. One participant mentioned, “I am trying to get to channels not on the screen. I looked and I saw there is a station called Disney – I said “next page” based on my previous experience with navigation.” Design Recommendation: Allow users to change the station via station name or number and use consistent voice command options throughout the system.

Task 3: Temperature Controls Participants felt there was hidden functionality with the temperature controls. For example, the majority of participants didn’t know there were multiple temperature zones in the car. However, participants liked the digital temperature display, the concept of the multiple zones, and the intuitive and simple nature of the knobs. To view all of the organized qualitative data for each sub-task, please refer to Appendix 6. Task: Using temperature controls Overall experience rating 7.7 out of 10 (+- 1.3)

Appeal rating based on usage during study

4 out of 5 (+- .8)

Task 3a: Syncing passenger and driver temperatures Only one participant was able to complete this task. The rest of the participants synced all four zones when trying to only sync the front zones. Participants expressed confusion about the lack of zone indication saying, “how am I supposed to know there are four zones, it’s obvious there are two”, “I would assume there are only two zones”, “I can’t tell that there are four zones”, and “I was looking for something to indicate that there are four zones, but based on what it looks like there are only two.” Design Recommendation: To minimize confusion, indicate the number of temperature control zones on the sync button with a number “4” under the word “sync”.

14

Task 3b: Syncing all four zones Most participants completed this task when trying to sync only the front two zones. Design Recommendation: To minimize confusion, indicate the number of temperature control zones on the sync button with a number “4” under the word “sync”.

Task 4: Touchpad Participants found the touchpad very interesting, calling it “cool” and “easy”. However, all participants wanted the touchpad to recognize more gestures. Furthermore, 7/9 participants were confused as to why the touchpad mode didn’t automatically change to allow them to scroll around the map when the MMI screen was showing the map, like how touchpad functionality automatically changed to accept letter inputs when the MMI screen showed the telephone directory. To view all of the organized qualitative data for each sub-task, please refer to Appendix 6. Task: Using the touchpad Overall experience rating 6.3 out of 10 (+- 1.9)

Appeal rating based on usage during study

3.7 out of 5 (+- 1.4)

Likelihood of use 3.8 out of 5 (+- 1.0)

Task 4a: Finding a contact in phone book All participants wanted and expected the touchpad to have more gesture recognition capabilities. One participant said, “I expected it to scroll up and down” while looking for a contact and, similarly, another participant expressed that he, “would want to scroll rather than put in a letter.” Design Recommendation: Implement scrolling capabilities for the touchpad to align its functionality with user’s mental model by.

Task 4b: Scroll around the map While performing this task, participant’s expressed confusion about the touchpad’s modality changes in relation to the MMI. Participant’s expected that the touchpad would control whatever is on the MMI. One participant mentioned, “I want to do radio things in radio mode and nav things in nav mode on the touchpad.” Another participant similarly commented, “If I were on navigation screen, the touchpad should change into navigation mode. I wouldn’t expect it to still be on the radio presets.” Design Recommendation: Have the touchpad change modes consistently (i.e. change

15

functionality to match what is on the MMI screen) or provide explicit feedback as to the current mode of the touchpad.

Overall Experience Data We used the SUS, NPS, and Plutchik’s emotion wheel to attain a holistic view of participant’s experience with the IVI system. The SUS is an industry standard for measuring the perceived usability of systems, NPS is used to measure brand loyalty, and the emotion wheel allows us to understand specific emotions related to user’s experiences. We found that the mean SUS score given by participants was 44.4/100 (+-22.5). This is much lower than a score of 68, which is considered to be the average SUS score (Figure 5) xv. Furthermore, the net promoter score for this system, on a scale from -100 to 100, was a -44.4, meaning that there were more detractors than promoters for the Audi IVI system. Even though these ratings are not positive, when participant’s reflected upon their emotions while using the system with the emotion wheel, there was a high response rate for strong feelings of interest and amusement. Participant’s strong feelings of interest and amusement show that this system is intriguing and fun for users, but that the system needs to become more usable. Having an IVI system whose information architecture better aligns with the user’s mental model will allow Audi to maintain and enhance their brand reputation and increase sales of Audi vehicles and their IVI systems. Design Recommendation: Through our findings we found several usability issues and listed the severity of each issue. By acting upon the data, Audi will be able to become the industry leaders in in-vehicle experience.

Figure 5: A visual representation of the results from the SUS

16

Conclusion and discussion

Conclusion By using both qualitative and quantitative data collection methods, we have been able to assess the usability of the 2013 Audi A6 IVI system. This study has revealed some usability issues that designers should act upon. The issues, along with their usability severity are as follows:

1. Voice commands, Severity: High a. Understand a variety of commands, not just one when attempting to

navigate to a location (e.g. “find”, “navigate”, and “get directions”). b. Use consistent command options (e.g. next page for radio voice

command options) throughout the system. c. When the user is attempting to call someone, the system should be

able to act upon first name only. Only if the user has more than one contact with the same first name should system prompt the user to say the last name of the contact he/she wishes to call.

2. Bluetooth connectivity, Severity: Medium a. To minimize confusion, provide step‐by‐step prompts for Bluetooth

syncing, including what features need to be activated on the phone, when the features need to be activated, and where to find these features (i.e. enable Bluetooth via the settings screen on the device and make sure pairing mode is on). If the phone is unable to sync, provide helpful and contextual feedback to the user.

3. Touchpad, Severity: Medium a. Align the touchpad’s functionality with the user mental model:

implement scrolling capabilities for the touchpad. b. Strive for modal consistency: have the touchpad change modes

consistently. 4. Temperature buttons, Severity: Low

a. Provide visible information as to the number of temperature control zones by placing the number “4” under the word “sync” on the sync button.

5. Screen expectations (touch screen), Severity: Low a. Consider having touch screen capabilities implemented in the MMI

screen. The projected growth in the IVI sector provides Audi with the opportunity to enter this rapidly developing market and become the industry leader in IVI systems in terms of usability and functionality. By fixing these usability issues, the SUS score of the Audi IVI system will increase from its current failing grade and its NPS will increase from its current negative score. From the emotion wheel, it is apparent that users find the system interesting. By making the system more user-friendly, users will desire only the Audi IVI system in their car. This will allow Audi to dominate the

17

growing field of IVI systems, increase profits, promote safety, and increase the Audi brand reputation.

Discussion Even though our study successfully uncovered some inconsistencies between user’s

mental models and the system model of the Audi IVI system, this study does have a

few shortcomings. The biggest shortcoming of this study is that users performed

the tasks in a stationary car. Users are supposed to use the IVI system while driving

but due to liability and safety concerns, this was not possible.

Furthermore, when the second participant attempted to sync his/her phone to the

car, four phones had already been saved to system. Because of this, the user had to

overwrite one. This was not something we had planned for and we had to tell the

participant to continue on with the task as though that hadn’t happened. To prevent

this from happening again, we reset the system to factory defaults in all of the

following studies. Similarly in one session, we forgot to set the radio presets, which

caused the radio station to not change when we asked the participant to change the

radio station using the presets. Additionally, after the first session, we changed the

way we asked participants to find directions to an address and a location to reduce

bias. We changed our command from, “Find directions to…” to “please get route

guidance to…” Also, we held one session in the evening and the participant

mentioned it was hard to see some buttons because he had turned the headlights off.

Finally, it was difficult to control certain variables while conducting the study. For

example, we had to be very focused to make sure that participants were making

voice commands from the same screen as one another. This is due to the fact that

the commands that appear on the MMI screen differ in various menus. Even with

these shortcomings, this study enabled us to find discrepancies between

participant’s cognitive model and the system model, allowing us to provide insight

into the usability of the 2013 Audi A6 IVI system to Audi executives.

18

Appendix 1: Screener Survey

Human Centered Design & Engineering Usability Test Screener

Project Description: Usability Study – Audi Infotainment Testing Location: University of Washington, Seattle Campus Testing Dates: November 11-16, 2013 Document Date: November 5, 2013

1. Do you or anyone in your household work in any of the following fields?

a. Advertising, Market Research, Public Relations

b. Journalism, newspaper, magazine publishing or professional blogging

c. TV or Online video

d. Auto design or sales

e. Don’t know-Terminate

2. What is your current employment status?

a. Full time

b. Part time

c. Student-Terminate

d. Retired

e. Stay at home partner/parent

f. Unemployed-Terminate

g. Other

19

3. What company do you work for? ______________________________

4. Job title: ____________________________________

5. How old are you?

a. Under 25-Terminate

b. 25 - 29

c. 30-35

d. 36-40

e. 41-45

f. 46- 50

g. 51-55

h. 56 or older

6. Do you have a valid driver’s license?

a. Yes

b. No-Terminate

7. Do you own a car?

a. Yes

b. No-Terminate

8. How frequently do you drive your primary car?

20

a. Everyday

b. 4-6 times/wk

c. 2-3 times/wk-Terminate

d. Once a week or less-Terminate

9. What is the year, make and model of the car you primarily use?

a. Year ____________________

b. Make ____________________

c. Model ___________________

d. Type ___________________ (No trucks or vans)

10. What type of technology does your car have integrated into it:

a. Navigation System

b. Bluetooth link to phone

c. Voice commands

d. Touch Screen input

e. Other: ________________

11. Do you have an app for your car on your phone?

a. Yes

a.i. What are you doing with it?

a.ii. Are you gathering any info remotely on your phone?

b. No

21

12. On an average driving day, I am in my car for:

a. 4+ hrs/day

b. 2-4 hrs/day

c. 1 hr/day or less-Hold

13. On an average driving day, I am driving my car to: (check all that apply)

a. Run personal errands

b. Drop off/pick up: (check any that apply)

d.i. family/friends

d.ii. clients/customers

c. Commute to work

d. Accomplish work-related tasks: (check any that apply)

d.iii. Meet clients

d.iv. Deliver products

d.v. Other __________________

Hold if participant only selects “C. Commute to work”

14. Does your in-vehicle system have any of the following components?

If yes:

i. iPod/MP3 Factory Aftermarket

ii. DVD w/ embedded screens Factory Aftermarket

iii. GPS/Navigation system Factory Aftermarket

iv. Onstar/other safety notification system Factory Aftermarket

v. Bluetooth Factory Aftermarket

vi. Sirius Factory Aftermarket

22

vii. Other Factory Aftermarket

iPod/MP3 only-Terminate

a. For each component mentioned above, how often do you use it?

a. Daily

b. 5-6 times/wk

c. 3-4 times/wk

d. 1-2 times/wk-Hold

e. Rarely-Terminate

f. Never-Terminate

15. How much do you like driving?

a. I love it, sometimes I go on a drive just for the fun of it.

b. I like it fine, it gets me from point A to B

c. I only drive when I need to, and avoid it when I can-Terminate

d. I prefer not to be the driver when in the car-Terminate

16. Do you own a smartphone (that you use internet, email, and apps on)?

a. Yes

b. No-Terminate

17. What kind of smartphone is it?

a. Android -> GOTO 20- 21

b. iPhone -> GOTO 22-23

23

Note: Recruit 50% Android and 50% iPhone

18. IF ANDROID OWNER: Do you use Google Now?

a. Yes

b. No

19. How often do you use Google Now in your car?

a. Never

b. A few times a month

c. A few times a week

d. Daily

20. IF iPHONE OWNER: Have you ever used Siri?

a. Yes

b. No

21. How often do you use Siri in your car?

a.a. Never -

a.b. A few times a month

a.c. A few times a week

a.d. Daily

22. How often do you synch your smartphone with your car?

a. Every time I drive

b. A few times a week

24

c. Rarely

d. Never

23. What smartphone apps do you use most often when you’re driving: either in your car, when planning a route, etc? (maps, yelp, podcasts, etc.)

a. ____________________

b. ____________________

c. ____________________

24. What is your household income?

a. Less than $20K-Terminate

b. $20 - $40K-Terminate

c. $40-60K

d. $60-100K

e. $100-125K

f. $125-150K

g. $150K +

25. Gender

a. Male

b. Female

Recruit a mix

26. Are you comfortable with being video recorded? a. Yes b. No

25

Appendix 2: Consent form Purpose:

Thank you for your interest in the study. We are students in the department of Human Centered Design and Engineering at the University Of Washington (HCDE). We are conducting a study involving the usability of Audi in-vehicle infotainment systems. Participants in this study will assist in providing crucial data that is used to improve the usability of in-vehicle infotainment systems. Evaluation Procedure and Risks: You will be asked to complete a series of tasks using the controls of the cars. While you speak, we will video record your actions and transcribe your comments. No information will be used outside of the purpose of this research study and will be used exclusively by the research team. In this study, we will ask you to complete a series of tasks and we will ask you to think-aloud as you perform each task. Note that the risks posed while performing these tasks, while minimal, do exist. The car’s engine will not be turned on, and you will never drive the car. However, since we are in a parking lot, there is a possibility that another vehicle could hit this car, as well as other dangers posed while sitting in a parked car in a public parking lot. Confidentiality: The purpose of this usability study is to evaluate the design and usability of the Audi in-Vehicle Infotainment System. Your input in the study and any findings will help improve this feature. This usability study may be recorded for the purpose of improving the in-vehicle infotainment system only; records will not be distributed in any shape or form. Breaks:

There will not be a scheduled break. However, you may take a break at any time. Freedom to Withdraw: You may withdraw from the study at any time without penalty. If you have any questions after today, please contact Emma Bulajewski (206-618-9687) or Sheena Kapur (971-998-5512). _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ I, ___________________________, have read and fully understand the extent of the study and any risks involved. I understand that the usability study may be recorded. All of my questions, if any, have been answered to my satisfaction. My signature below

26

acknowledges my understanding of the information provided in this form and indicates my willingness to participate in this user testing session and not hold the research team, Audi, or any of its affiliates responsible for any harm that I may endure during this study. Signature: __________________________________________________________ Printed name: _______________________________________________________ Date: _______________________________

27

Appendix 3: Participant Questionnaire

Task Evaluation

Syncing the phone 1. How was your experience of setting up/connecting your Smartphone to the car?

Not at all positive 1 2 3 4 5 6 7 8 9 10 Extremely positive What is the most important reason for the score you gave? 2. Based on your experience setting up “hands-free” capabilities with your car here today, how appealing is this technology to you?

5. Extremely appealing

4. Very appealing

3. Somewhat appealing

2. Not very appealing

1. Not at all appealing

*Appeal: the extent to which you would desire this technology

What specifically did you like about the process of connecting your Smartphone with the Audi car? (Please be as specific as possible.) What specifically did you not like about the process? (Please be as specific as possible.) How would you describe the process of setting up your phone to the Audi to a colleague or friend? What would you tell them about it? (Please be as specific as

28

possible.) Based on your set up experience, how likely would you be to connect your Smartphone to your Audi? (CIRCLE ONE)

5. Extremely likely

4. Very likely

3. Somewhat likely

2. Not very likely

1. Not at all likely

How would you improve the process of connecting your Smartphone to your Audi? What, specifically, would you recommend? (Please be as specific as possible.)

Please do not turn to the next page until instructed to do so.

29

Voice Commands 1. How was your experience of using voice commands with the Audi?

Not at all positive 1 2 3 4 5 6 7 8 9 10 Extremely positive What is the most important reason for the score you gave? 2. Based on your experience of using voice commands with the Audi here today, how appealing is this technology to you?

5. Extremely appealing

4. Very appealing

3. Somewhat appealing

2. Not very appealing

1. Not at all appealing

*Appeal: the extent to which you would desire this technology What specifically did you like about your experience of using voice commands with the Audi? (Please be as specific as possible.) What specifically did you not like about the experience? (Please be as specific as possible.) How would you describe the experience of using voice commands with the Audi to a colleague or friend? What would you tell them about it? (Please be as specific as possible.)

30

Based on your set up experience, how likely would you be to use voice commands with your Audi? (CIRCLE ONE)

5. Extremely likely

4. Very likely

3. Somewhat likely

2. Not very likely

1. Not at all likely

How would you improve the experience of using voice commands with the Audi? What, specifically, would you recommend? (Please be as specific as possible.)

Please do not turn to the next page until instructed to do so.

31

Infotainment Controls 1. How was your experience of using the buttons to control the temperature in your Audi?

Not at all positive 1 2 3 4 5 6 7 8 9 10 Extremely positive What is the most important reason for the score you gave? 2. Based on your experience setting the temperature in your Audi with your car here today, how appealing is this technology to you?

5. Extremely appealing

4. Very appealing

3. Somewhat appealing

2. Not very appealing

1. Not at all appealing

*Appeal: the extent to which you would desire this technology

What specifically did you like about the process of to setting the temperature in your Audi? (Please be as specific as possible.) What specifically did you not like about the process? (Please be as specific as possible.) How would you describe the process of setting the temperature in your Audi to a colleague or friend? What would you tell them about it? (Please be as specific as possible.)

32

How would you improve the process of setting the temperature in your Audi? What, specifically, would you recommend? (Please be as specific as possible.)

Please do not turn to the next page until instructed to do so.

33

Touchpad 1. How was your experience of using the Audi touchpad?

Not at all positive 1 2 3 4 5 6 7 8 9 10 Extremely positive What is the most important reason for the score you gave? 2. Based on your experience of using the Audi touchpad, how appealing is this technology to you?

5. Extremely appealing

4. Very appealing

3. Somewhat appealing

2. Not very appealing

1. Not at all appealing

*Appeal: the extent to which you would desire this technology

What specifically did you like about of using the Audi touchpad? (Please be as specific as possible.) What specifically did you not like about the process? (Please be as specific as possible.) How would you describe the Audi touchpad to a colleague or friend? What would you tell them about it? (Please be as specific as possible.)

34

Based on your experience of using the Audi touchpad, how likely would you be to use this technology? (CIRCLE ONE)

5. Extremely likely

4. Very likely

3. Somewhat likely

2. Not very likely

1. Not at all likely

How would you improve the process of using the Audi touchpad? What, specifically, would you recommend? (Please be as specific as possible.)

Please do not turn to the next page until instructed to do so.

35

Post-Test Questionnaire How likely, if at all, are you to recommend Audi to a colleague or friend? Please consider the scale from 0 to 10 where “0” means “very unlikely” and “10” means “very likely.” (CIRCLE ONE)

0 1 2 3 4 5 6 7 8 9 10

Very Unlikely Very Likely

Based on your entire experience using the Audi system, please fill out the following emotion wheel. The larger circles indicate stronger feelings, while the smaller circles indicate not as strong of feelings.

36

Please fill out the following scale, again, based on your entire experience. If you are unable to respond to a particular item, please mark the center of the point scale.

37

Appendix 4: Facilitator script

Facilitator Greeting Hi I’m [name] and I’ll be the facilitator for this study. Thank you so much for spending your time to help us! We really appreciate your input. Your feedback will help us get a better idea about the usability of in-vehicle infotainment systems. Test Facilitator’s Role We will record your reactions and opinions; so, we may ask you to clarify statements that you make from time to time. [NAME] is our note-taker and will record your reactions and comments. James is our technical expert and is here in the event of a technical issue. Please feel free to ask me any questions you have throughout the test. During this session, however, I will not be able to offer any suggestions or hints. There also may be times when I’ll ask you to explain why you said or did something. After the test, I will ask you some questions about the IVI and your opinions and suggestions. Test Participant’s Role I will be asking you to perform a series of tasks in order to learn if it works well for you. We’ll do this by giving you tasks to complete, asking you to “think out loud” while you perform these tasks, and then give you surveys to fill out after you complete these tasks. You also will be asked a series of questions about your experience at the end of this session. Things to keep in mind Before we get into our discussion, I need to go over some housekeeping instructions:

This session will last approximately an hour and a half. There will be observers during this study. This is not a test of you; you’re testing the system, so don’t worry about

making mistakes or letting us down. If you ever feel that you are lost or cannot complete a scenario with the

information that you have been given, please let me know. I’ll ask you what you might do in a real-world setting and then either put you on the right track or move you onto the next scenario.

Also, I want to really encourage you to “think-out loud” while you perform tasks: this will allow us to understand your though processes as you go through the tasks we give you.

And I want to stress the importance of candid and honest answers o As we have this discussion, we will have you complete a questionnaire.

Please make sure to complete all questions on the survey and answer the open-end questions as thoroughly as possible.

o If at any time you’re not sure about any question or how to use the rating scale, please ask me immediately. As we have this discussion,

38

we will have you complete a questionnaire. Please make sure to complete all questions on the survey and answer the open-end questions as thoroughly as possible.

Do you have any questions for me so far? [If no, move on.] Here is the consent form for you to sign to inform you of your rights.

[Look over form]. Ok, let’s get started. We’ll begin with a brief tutorial of some of the key features of the Audi system that you will be using.

Tutorial This is the home screen for the infotainment system [point to screen]. Each of these sub categories can be accessed using this rotary dial. The settings button for each of these categories is right here [reference settings button]. Does this make sense? If yes, move on. If no, explain again. Push to talk function: The push-to-talk function allows you to control your infotainment system in a “hands-free” way. Use the push-to-talk feature, you push this button [push button] and wait for the beep [listen for beep]. Does that make sense? If yes, move on. If no, explain again. The touchpad: This is the touchpad. It recognizes motion when touching it and can be used as an alternative to some buttons. For example, instead of using the rotary dial to fins someone in your phonebook [show them] you can use the touchpad [make sure NOT to show them, as this is a task]. Does this make sense? If yes, move on. If no, explain again.

Tasks Ok, so let’s get started with the first task. Please make sure Bluetooth is turned on for this task and keep your phone awake. First, I would like to begin by asking you to:

Sync your phone to the car for Bluetooth, hands-free, calling capabilities. [When the participant gets to the point when they are about to sync their phone with the car, make sure they hit the “connect” button on their phone]. Now that you’ve completed syncing your phone please fill out this survey. [Look over results and ask any follow up questions if necessary.] Great, now that you’re done filling that out, let’s move on to a few tasks using voice commands. First:

Please get route guidance to 5038 17th Ave NE, Seattle, Washington.

39

Call a specific person in your phonebook. Please get route guidance to “Pagliacci Pizza”. Change Sirius radio to a different Sirius radio channel.

Great. Please fill out this survey regarding your experience using voice commands. Now let’s move on to some tactile tasks using the infotainment controls. Please use the infotainment controls to:

Input the driver side climate control settings to a pleasant 72 degrees F. Sync the driver side temperature control to the passenger temperature

control setting. Sync all 4 temperature zones of the car.

Awesome. Please fill out this survey now. [Look over results and ask any follow up questions if necessary.] Let’s move on to the last of the tasks we are testing today using the touchpad. Please use the touchpad to:

Find a person in your phonebook. o Complete o Incomplete

Change a radio preset. o Complete o Incomplete

Scroll/pan the map on the navigation screen. o Complete o Incomplete

Great, here’s a survey to fill out about your experience using the touchpad. [Look over results and ask any follow up questions if necessary.] Alright, we’re done with the tasks. Now we’re going to ask you to fill out a survey about your entire experience. Great, now please fill out the following scale, again, based on your entire experience. If you are unable to respond to a particular item, please mark the center of the point scale

Thank and Wrap-Up Let me review this and see if I have any further questions. Look through participant’s answers for completeness. Do you have any additional information you’d like to share? Do you have any thoughts on the study?

40

Do you have any questions for me? Thank you so much for your assistance today.

41

Appendix 5: Survey data Survey data: https://docs.google.com/a/uw.edu/spreadsheet/ccc?key=0AsB2oIx4MDOsdERyUm4za044OTVaZm1GQ1BwaWVlcHc&usp=drive_web#gid=0

Appendix 6: Analyzed qualitative data Organized qualitative data: https://docs.google.com/a/uw.edu/document/d/14TvEkDbl6JWFVHs-bksYDIEk3C91p-OvDkD3fhwY0Bw/edit?pli=1

Appendix 6: Raw notes

Participant 1 Notes for first task (Synching phone): Task completed (Y/N) Task Time (sec) Y 3:54 Bluetooth is on. Device is pairing. “Wondering if I have to turn it on in the car?”, “Not getting any number… is it paring? Should I have hit them at the same time? “Confused” Trying again. Nothing happening… tap tap tap. “Not sure what to do now”, “maybe try turning on and off”, “I’m assuming it’s working… oh no, it didn’t work.” “Interesting… I’ve paired with other items in the past easily.” “I don’t know if it’s loaded my contacts on to the device, it’s not clear. No message to let me know that it is loading contacts/apps etc.” Would you like this feature? “I’d like to know that that’s what it’s doing.” Notes for finding directions to 5038 17th ave NE: Task completed (Y/N) Task Time (sec) Y 2.37 Said it slowly, and the voice command prompt cut him off. Then repeated it fast. 3 attempts. Disliked paused for the beep. I have to pay attention for the beep, how long of a window do I have? I want to touch it (the IVI screen). Notes for calling a specific person in your phonebook. Task completed (Y/N) Task Time (sec) y 1:37 3 attempts. First 2 attempts tried first name only. Then tried first and last name. Frustrated. “No, No , NO!” The number is being dialed even after he said “no”. (we all laugh) Notes for getting directions to “Pagliacci Pizza”. Task completed (Y/N) Task Time (sec)

42

Y 5:21 Forgot to push the button. “The Jukebox?” “Sigh… how did I get to phone from pizza? AMAZING!” “CANCEL!” “I’m getting irritated with it” “Really cancel, I mean it!” “Oh we’re back to this screen…. CANCEL” “Do I have to yell at it?” finally figured out “Online destination” It didn’t match other things I have used, other rentals, didn’t have a built in POI, I didn’t understand what it was doing. I want to search by name and not go so many steps. Not congruent with other experiences, would have to learn this system specifically. Maybe a demo mode first? I’m sure there’s a manual, but who reads the manual. Notes for changing Sirius radio to a different channel. Task completed (Y/N) Task Time (sec) Y 1:39 Follow up questions notes: “That was a command earlier” chose help. “Cancel” “Sigh”. Tried to use line commands that worked in the navigation setting on help screen, which did not work. I wanted to go to specific radio station. Not very responsive, didn’t understand most of my commands. Seemed much slower than entering manually. Did some unexpected things, when I said cancel and it wouldn’t go anywhere. Did cancel cancel? Not very appealing, it was like a puzzle… and I had to read it? It never confirmed my commands, and had to read the screen anyhow! Tell me what it’s doing, need feedback. Thinks you have to be in certain category (e.g. telephone) to use a telephone command. Notes for inputting the driver side climate control settings to a pleasant 72 degrees F. Task completed (Y/N) Task Time (sec) y 0:13.01 Follow up questions notes: Notes for syncing the driver side temperature control to the passenger temperature control setting. Task completed (Y/N) Task Time (sec) N 53 Follow up questions notes: Synced all 4. Turned AC on and off, tried to reset. I like the sync button, it sounds good. “I wish it were a touch screen!” “No clue how to sync” “I have no sign that there are different temperature zones”

43

Notes for syncing all 4 temperature zones of the car. Task completed (Y/N) Task Time (sec) Y 0:14 Follow up questions notes: Touchpad: find a person in phonebook Task completed (Y/N) Task Time (sec) y 1:19 Follow up questions notes: Trying to swipe. “Oh it wants me to write letters?” “Whacky!” “Who is that??” “How did you get to Q?” “Oh that’s delete?” “Huh” “I’m going to hit the back button””ph it takes it slowly” I’m used to swiping through a list. More tempted to use a voice. I wouldn’t have to look, I like it. Once I understood what it was trying to do, Touchpad: change to radio preset Task completed (Y/N) Task Time (sec) y 0:06 Follow up questions notes: I tapped the number, but I had to look way down here. I would have to look at an entirely different place. Kind of awkward down there. I like it more accessible by my hands, I’m supposed to be driving! If I drove this I might realize and get used to the tech. I like the spinny thing (on the steering wheel)! Likes input controls to be near his hands. Doesn’t want to have to reach further from normal driving position (down to where the rouchpad is versus on the infotainment panel). Touchpad: Select a spot on the map and zoom around using the touchpad. Task completed (Y/N) Task Time (sec) N 2:48 Follow up questions notes: I would like to use touch pad to touch and drag. These buttons don’t do what I want. HMM. Still doesn’t work. Can’t touch the screen. Huh. There’s no label. (off track not using touchpad) Confused. I’d just zoom out or get out the manual. I actually liked the touch pad, cause I could touch it lightly. Fits where my arm is. It lights up to tell me I can do stuff. Appealing. Bit of a learning curve. I don’t like that I can’t navigate with it… not just select. I’m not sure of what it can do. I wanted to be able to use two fingers like on an iPhone. Interesting and has promise. Very likely to use this technology. Would like it to have multi-functionality. A bit unnecessarily complex.

44

This is where my eyes are going, it would be nice if my hands went there, too. I want to be engaging the thing in front of me. I like that it’s mirrored over here. The head’s up display in the Camero. The final likert questions were a bit biased. These questions propose some negative feedback! CHANGE CARD: Use the voice command to get route guidance to: address. Did not bias our participant, he said “Navigation” first. Can see how this might introduce bias. Ended 1:03

Participant 2 Notes for first task (Synching phone): Task completed (Y/N) Task Time (min/sec) Y 5:10 I don’t use Bluetooth currently. I should turn it on. I assume that’s all I do. Fail. She’s going to the website…. Searching for directions on how to sync bluetooth on her phone. 404 error. This is very annoying… I might ask someone or call Audi. It worked. It should prompt me that it can’t connect to MMI. Notes for finding directions to 5038 17th ave NE: Task completed (Y/N) Task Time (min/sec) N (found this when she was on next task) 6:00 Kept saying “locate”. Should be able to recognize an address. I don’t want to indicate that it’s not a contact. Okay so now I hate this and I will tell my friends that. There should be commands that tell me what to say. I don’t want to have to relearn everything. Why can’t it be the same as my phone function. Realistically I would use manual entry. Is there something I need to do to activate navigation? Failed Notes for calling a specific person in your phonebook. Task completed (Y/N) Task Time (sec) N She said “Mom” It went to map. Interesting. Oh now she is doing the previous task. I shouldn’t have to say the full name. It worked. Keeps clicking and talking, not saying the line. This is really silly. It does not understand the way I speak. At this point I might punch it. Maybe it needs to learn my voice better? The point of this device is to make me safer and the task easier. Notes for getting directions to “Pagliacci Pizza”. Task completed (Y/N) Task Time (sec) N She said “online destination”… and it took her to the address entry screen. She spoke before the beep. I’m pretty frustrated. I would stop using voice altogether, it’s

45

inefficient. Notes for changing Sirius radio to a different channel. Task completed (Y/N) Task Time (sec) N That’s not what I thought it would do, I thought it would choose the station 6, and not line 6. I expected I could say the name of the radio station or the number. Follow up questions notes: It seems on but I am tunring on and off there. The buttons do not meet mental model. Notes for inputting the driver side climate control settings to a pleasant 72 degrees F. Task completed (Y/N) Task Time (sec) y 0:02 Follow up questions notes: Notes for syncing the driver side temperature control to the passenger temperature control setting. Task completed (Y/N) Task Time (sec) N Follow up questions notes: Synced all 4. I would never know I am syncing the back as there is no dial that tells me. Notes for syncing all 4 temperature zones of the car. Task completed (Y/N) Task Time (sec) Follow up questions notes: Touchpad: find a person in phonebook Task completed (Y/N) Task Time (sec) N Is trying to scroll. What do you do with this thing? It wants me to put in letters. I though it would be like a dial/mouse. Typed M and didn’t go to the mom contact, she forgot to delete the previous letters. Mom is a short name and if I have to type a whole name while driving I would not be so stoked. That was supposed to be a J. You have to be really specific. I would give up and then scroll to J’s, much faster Follow up questions notes:

46

Touchpad: change to radio preset Task completed (Y/N) Task Time (sec) Y 0:01 I would have never guessed without the tutorial. Follow up questions notes: Touchpad: Select a spot on the map and zoom around using the touchpad. Task completed (Y/N) Task Time (sec) Y 3:32 Would like to use dial as driver stick. I wish this was a touch screens. There’s a disconnect between knobs and the screen. I’m looking for a sign like “scroll” but don’t see it. Oh there it is, but kept using the button and couldn’t select. Finally saw the touchpad icon and figured it out. “Shouldn’t have to switch between buttons and touchpad.“ Multi-purpose functions. “You wanted to go out for coffee? Sorry, that will be in Chinese.” Touchpad seems silly unless you can do everything on it. I want to be able to do all tasks with each feature. Follow up questions notes: Would not recommend to friends to buy Audi with the information system. Would suggest getting 3rd party navigation. Expects that she would always be frustrated with the system. Would regret if I purchased this system and it worked like this. Seems they should have the system working before it goes out on market. I already know there are way better systems, I’m not wowed and Audi is behind the times.

Participant 3 Notes for first task (Synching phone): Task completed (Y/N) Task Time (sec) N Follow up questions notes: I’m a bit confused as to why it wouldn’t pair up. I’m checking for more options, but probably I would talk to the dealer, as I can’t see a way to make it work. The car apparently wasn’t visible to my phone. I expected it would be visible from either direction. Notes for finding directions to 5038 17th ave NE: Task completed (Y/N) Task Time (sec) Y 6:30 Follow up questions notes: That was pretty crazy. It was frustrating that at first it didn’t tell me my options, it didn’t say enter city or zip. I had to enter in reverse. There’s no indication that I can remember on whether I should enter the zip code. Notes for calling a specific person in your phonebook.

47

Task completed (Y/N) Task Time (sec) Y 00:37 Follow up questions notes: It didn’t know which one of the two contacts had a phone number or not. I didn’t like that it makes you look at the screen to pick a line number, it should ask if you want to call home, work. Etc. There was no point in showing me the first contact with no number. Notes for getting directions to “Pagliacci Pizza”. Task completed (Y/N) Task Time (sec) Y 3:43 Follow up questions notes: I don’t see how to put in a name. Said Help Command: “That wasn’t very useful!” How do I enter it in? Confused. (Finally found it by browsing – “next page”) That was kind of horrible. There should be standard button that takes me back a screen or cancels. Notes for changing Sirius radio to a different channel. Task completed (Y/N) Task Time (sec) Not really Follow up questions notes: What if I want to go to a radio station not on the screen? Interesting. I am trying to get to channels not on the screen. I looked and I saw there is a station called Disney – I said “next page” based on my previous experience with navigation. I don’t see from here how to find a channel if I don’t know the line number. “Interesting, annoying”. From here I can’t even see the frequencies. I don’t see a way to get to a station on a different screen. Notes for inputting the driver side climate control settings to a pleasant 72 degrees F. Task completed (Y/N) Task Time (sec) Y 0:05 Follow up questions notes: Notes for syncing the driver side temperature control to the passenger temperature control setting. Task completed (Y/N) Task Time (sec) N

48

Follow up questions notes: I’m not seeing anyway to do this. Not sure what that did… no I guess I can’t see how. I was looking for something to indicate that there are 4 zones, but based on what it looks like there are only 2. I didn’t think about what auto means. Once you get it it makes sense, but I didn’t immediately get it. Notes for syncing all 4 temperature zones of the car. Task completed (Y/N) Task Time (sec) Y 0:04 Follow up questions notes: Touchpad: find a person in phonebook Task completed (Y/N) Task Time (sec) Y 1:45 Follow up questions notes: What the heck is that doing? I would expect that it would put a mouse on the screen and would click on find contact. The question is how to cancel to start over. Its aleady searching based on my first entry. I would use the dial to select the phone number… I can’t see how to start the call with the touchpad… it should work for scrolling and up and down. Once I realized there might be a way to delete with the touch pad, I figured it out pretty quick Touchpad: change to radio preset Task completed (Y/N) Task Time (sec) Y 0:19 Follow up questions notes: It looks like when I am at this screen the touchpad is my radio preset. It seemed like at first it wasn’t changing anything. I had to be very deliberate, if I do it too quickly it doesn’t pick it up. If there is other movement it doesn’t register. Touchpad: Select a spot on the map and zoom around using the touchpad. Task completed (Y/N) Task Time (sec) N Follow up questions notes: (He tends to spend a lot of time going through the options with the scroll) I was looking for a control that lets me pan around. If I thought I could pan, I would have expected to use the touchpad. Looking in the menu that put the touchpad in a panning mode. Hmmm. I’m not seeing anyway to tell it to pan around. If I were on navigation screen, the touchpad should change into navigation mode. I wouldn’t expect it to still be on the radio presets.

49

Last comments: In general, fro the UI point of view it’s not quite mature. Especially the voice system. The usability aspect is not easy to use and somewhat immature. Great to figure out how to make it more usable.

Participant 4 Notes for first task (Synching phone): Task completed (Y/N) Task Time (sec) y 2:21 Follow up questions notes: Bluetooth is clunky to set up. Should be easier. I would rather it had an option to play music, than just loading contacts. Didn’t know what it was doing – beach ball? No feedback about where it is at in the process. Is it going to work? Would like to see an indication of modes. Notes for finding directions to 5038 17th ave NE: Task completed (Y/N) Task Time (sec) Y 1:31 Follow up questions notes: It would be nice if it listed clear instructions for finding an address. It’s not clear how to enter address. Info I receive about the process is poor. Notes for calling a specific person in your phonebook. Task completed (Y/N) Task Time (sec) Y 0:45 Follow up questions notes: Notes for getting directions to “Pagliacci Pizza”. Task completed (Y/N) Task Time (sec) Y 0:58 Follow up questions notes: Can I get it to say ROWT instead of ROOT? That would be annoying as shit. Notes for changing Sirius radio to a different channel. Task completed (Y/N) Task Time (sec) Y 1:08 Follow up questions notes: It’s confusing. Not sure how to set or get to a station I actually want, I want to be able to say “Sirius station 094”, I have the assumption that I can speak naturally, so there’s a bit of an intellectual disconnect. Notes for inputting the driver side climate control settings to a pleasant 72 degrees F.

50

Task completed (Y/N) Task Time (sec) Y 0:05 Follow up questions notes: Notes for syncing the driver side temperature control to the passenger temperature control setting. Task completed (Y/N) Task Time (sec) Y 0:25 Follow up questions notes: There was a lag between syncing so I was confused about what it was doing and then suddenly they were synced. It took me a minute to figure out what had happened. Probably this was to prevent accidental clicks. Notes for syncing all 4 temperature zones of the car. Task completed (Y/N) Task Time (sec) Y 1:25 Follow up questions notes: No feedback that there are 4 zones in the car (tried turning knobs in back seat zone to look for feedback – none found) Ahh the sync button. Why is that button no lit up? Mod: What aspect was shit? Car is too warm and there are too many ways to set the temp. On the plus side you can do everything, on the downside you have to deal with everything, Touchpad: find a person in phonebook Task completed (Y/N) Task Time (sec) Y 0:30 Follow up questions notes: I expected it to scroll up and down. Touchpad: change to radio preset Task completed (Y/N) Task Time (sec) Y 0:48 Follow up questions notes: Oh I didn’t expect that! (while on nav screen) If I clicked 2 I would expect to go to Missed Calls. Instinctively I would select radio and then go to the preset. I want to do radio things in radio mode and nav things in nav mode on the touchpad. Touchpad: Select a spot on the map and zoom around using the touchpad. Task completed (Y/N) Task Time (sec) Y 0:44

51

Follow up questions notes: I’m not sure why the radio options disappeared. For scrolling I would like a joystick function. Things like these are aggravating. Last comments: Touchscreen would be great. Need to learn special alphabet ie. I vs 1. Are there special things I can do with the touchpad? I want to know the preconfigured functions.

Participant 5 Mapping of rotary buttons not clear during tutorial Notes for first task (Synching phone): Task completed (Y/N) Task Time (sec) N Follow up questions notes: I’ve never used Bluetooth and have never needed it. I don’t want to talk and drive. I’m not sure I can touch it, can I touch the screen? Oh I have to use the buttons. I think when you’re driving this would be dangerous. Once you get used to it, it might be safe? I don’t know. I was hitting that button, but maybe I needed to do something with my phone. I don’t know where to enter a pin? I’m going to try again. Where is Bluetooth on the main menu? I tried and failed and nothing is showing up. I wouldn’t use it, I would give up. I don’t know if the problem is the car or my phone. Step by step… intuitive would be to go to “Car” and then into Bluetooth setup. If the car could tell me how to set it up. Notes for finding directions to 5038 17th ave NE: Task completed (Y/N) Task Time (sec) Y 1:22 Follow up questions notes: It might be my accent! (New Zealand) Notes for calling a specific person in your phonebook. Task completed (Y/N) Task Time (sec) Y 3:43 Follow up questions notes: (She adjusted her accent to sound more American). Instead of saying “End” it should go back to main menu. Saying “no I don’t want help” Doesn’t mean I want to end it. (tried to interrupt the help, it kept talking) I don’t know how to cancel!! Why is she not saying end like she did 2 minutes ago? I would pull over and use my phone. Yay it worked. I think my accent is part of the

52

problem. If I could configure it to understand my accent, I would really like that. You should be able to hit it again and turn it off to cancel. Notes for getting directions to “Pagliacci Pizza”. Task completed (Y/N) Task Time (sec) N Follow up questions notes: If this happened I would just use my phone, but if it did this 2 or 3 more times I would never use it. (Finished the task using rotary) Notes for changing Sirius radio to a different channel. Task completed (Y/N) Task Time (sec) Y 0:14 Follow up questions notes: Notes for inputting the driver side climate control settings to a pleasant 72 degrees F. Task completed (Y/N) Task Time (sec) Y 0:10 Follow up questions notes: Notes for syncing the driver side temperature control to the passenger temperature control setting. Task completed (Y/N) Task Time (sec) N Follow up questions notes: I reached over because I didn’t think that that would be reading this. (I want to do the action and then input sync myself, but now I see that. I can’t tell that there are 4 zones. It’s not telling me if it’s syncing all 4 or just 2. I don’t know. I would just move them one at a time. I would let the passenger figure out whatever temp they want. Notes for syncing all 4 temperature zones of the car. Task completed (Y/N) Task Time (sec) Y 0:16 Follow up questions notes: I don’t find the sync feature useful. One temp for a car is fine. The seat warmers are great. Touchpad: find a person in phonebook Task completed (Y/N) Task Time (sec) Y 1:54

53

Follow up questions notes: I was touching to figure out and surprised it drew a letter. How do I scroll down? How do I delete J and E? A cross? No. I want to call Eli. I would think the 4 corners would open more options. I would want to scroll rather than put in a letter. Oh I don’t want to call this person, how do I go back? I was trying to figure out delete was, and you showed me back so I figured it out. Even thought I put in A back would be better, delete sounds like you’re deleting the contact. Now how do I call? Touchpad: change to radio preset Task completed (Y/N) Task Time (sec) Y 0:43 Follow up questions notes: I see the numbers, but I didn’t realize they were related to the radio. I expected I would scroll through the homescreen icons. I like having the numbers there, now that I know. Touchpad: Select a spot on the map and zoom around using the touchpad. Task completed (Y/N) Task Time (sec) N Follow up questions notes: First I need to get it off radio. How do I get it off radio? I don’t know how to get it off the radio. Realistically I would want to touch it. Turning the knob doesn’t give me scroll options, so I don’t know. I would just use my phone. I have no idea how to get the touchpad off radio. If I were on the map, the touchpad should be off radio, that’s the biggest issue. If I owned this car, I would spend time getting to know it. It’s not as intuitive as I would like. The radio/touchpad is really annoying. Being able to customize the voice would be important. It’s okay. Maybe holding down the touchpad down would change the radio, why should I control the touchpad with the knob.

Participant 6 Notes for first task (Synching phone): I just went to my Bluetooth settings and clicked. It’s now searching. Reading prompt on phone. Hasn’t used the infotainment system yet. Looks at screen only after prompted from phone. Task completed (Y/N) Task Time (sec) Y 110 Follow up questions notes: Notes for finding directions to 5038 17th ave NE: Trying to say say “map to..”, “Navigation [address].”. Trying to say command all at

54

once. Said “map”, went to map. Just reads address. Uses navigation on Phone not in car. Task completed (Y/N) Task Time (sec) N N/A Follow up questions notes: Notes for calling a specific person in your phonebook. Knows how to do previous task because system went into help by accident Task completed (Y/N) Task Time (sec) Y 33 Follow up questions notes: Notes for getting directions to “Pagliacci Pizza”. “Map-àdestination addressàPagliacci pizza”. “Map Pagliacci Pizza”. He would have gone to the phone. Thinks when the map pops up with an arrow, that’s where Pagliacci is. Task completed (Y/N) Task Time (sec) N Follow up questions notes: Notes for changing Sirius radio to a different channel. Tries to say station name. “Why does she hate me?” Would tend towards executing tasks with buttons. Stopped task to comment. Would have used scrolling function by now. Task completed (Y/N) Task Time (sec) N Follow up questions notes: Notes for inputting the driver side climate control settings to a pleasant 72 degrees F. Asked for follow up for def of infotainment Task completed (Y/N) Task Time (sec) Y 26 Follow up questions notes: Notes for syncing the driver side temperature control to the passenger temperature control setting. Just reached over and changed it. “There’s nothing on there telling me these will sync” Task completed (Y/N) Task Time (sec) N

55

Follow up questions notes: Notes for syncing all 4 temperature zones of the car. Completed this task in previous task Task completed (Y/N) Task Time (sec) Y 8 Follow up questions notes: It feels very similar to my car, Toyota. More options though but the screen is similar. Touchpad: find a person in phonebook Because of the way Apple does it I would try and finger scroll. Oh, it’s doing it by letter. That was pretty easy to discover. So the one thing I would want to know, oh aha delete. I would probably be tempted to touch it (the screen). Not sure how I would call, oh tap. It (the screen) does look very touchable. Task completed (Y/N) Task Time (sec) Y 108 Follow up questions notes: Touchpad: change to radio preset Task completed (Y/N) Task Time (sec) Y 3 Follow up questions notes: Touchpad: Select a spot on the map and zoom around using the touchpad. It’s still in radio still. Oh whoa that’s not what I wanted (clicked button on steering wheel). Clicked steering wheel button due to his car. Tries the four buttons around the rotary knob. I’m on the nav setting and it’s still on the radio because I see the numbers. Yea, again, not being a car map user, I’d probably be driving. Realistically he’d try to touch the screen. Is able to pan but doesn’t know it. He feels he stuck in a mode where he can’t use it. Oh I see the icon (touchpad Icon) here so now I think I can scroll, yep. Finally found it. Wasn’t instantly discoverable. Once he found out how to do it, he wouldn’t forget though. Task completed (Y/N) Task Time (sec) Y 205 Follow up questions notes: End follow up: (About the screen). I really would want this to be a touchpad. If I were making this car, we have all of these (buttons) to make it easy to control, I would go the extra step and make this a touchpad. I expect myself for things to be touchscreen even when they’re not. I’d probably do that for a month before the muscle memory told me it wouldn’t work. Also, my car has the touch. So when I’m in trouble I can just reach over and touch it.

56

When asked about if control were intuitive: Well if this were a touchscreen I would use these (button) with the touchscreen. Um and I’m curious about what the Google part is (on the map). I’m curious about the other features of the car.

Participant 8 Notes for first task (Synching phone): Task completed (Y/N) Task Time (sec) Y 2:18 Follow up questions notes: I;m curious if the car is advertising on bluetooth. Hmm. Going to guess, telephone. Tthat’s problematic, why can’t you find my phone? It’s on for me. I’m curious if they are both looking why don’t they find each other. I should have been reading… it says on the phone that Bluetooth is not visible… more of a problem with my phone. I’m curious what it is asking permission to do… Having a hard time saying how positive this way. Any problems were a result of me not using my phone correctly. The system seems straightforward. I don’t know how the car could make that better. Maybe people who sync things more often have less trouble. It was an irritating process, but now that I understand on my device it was fine. I wonder if it’s possible for the IVI to detect you are having trouble and suggest help. Maybe it supposed to be so straightforward you don’t need help. Notes for finding directions to 5038 17th ave NE: Task completed (Y/N) Task Time (sec) Y 2:55 Follow up questions notes: “the list is being loaded” What does that mean? That was hard. It seemed to listen better once I remember to wait for the beep. Notes for calling a specific person in your phonebook. Task completed (Y/N) Task Time (sec) Y 1:05 Follow up questions notes: Said yes before beep, it went to map. “really”. That was difficult! I’ve ben using Voice on my a lot lately and it’s been easy as the recognition is natural. I don’t have to remember commands, I can just speak. It didn’t register when it was listening with the beep, if it didn’t hear me I wasn’t told nothing happened. If I were driving I would give up or use my phone. Notes for getting directions to “Pagliacci Pizza”. Task completed (Y/N) Task Time (sec) Y 4:10

57

Follow up questions notes: (Said “next restaurant”. ) (chose help) (enter address: not what I want) I feel like I am missing something or not seeing where I can tell it a place. I don’t understand what online destinations means. I’m not sure what any of it means. I would use my phone right now. I;m not expecting it to have natural recognition, so I am going off the prompt. I guess I could say “paglicacci Pizza”, nope. No, cancel. (Ran into data connection issue) . That was really hard! The options presented don’t seem intuitive. What does Online Destination mean? I can’t fathom. Now I understand it will use my phone’s data to search for something in the vicinity. I would have expected more general language searching. Less intuitive than other voice systems I have used. If this was my experience talking to a device to get it to do stuff, I would not care. It was really hard. I understand I am diving in without reading, I would have read the manual. It was hard to figure out when it was listening to me. It’s less natural. Seems to work really well in it’s own language, but the only voice I have used and you just talk to it and it figures out. As I got used to it I would use it more. There was a point where it tried to help and offered a list of help optins, but it was so clunky. I would probably pull over, rather than pay attention to what the computer read to me from the list. Notes for changing Sirius radio to a different channel. Task completed (Y/N) Task Time (sec) Y 0:10 Follow up questions notes: Notes for inputting the driver side climate control settings to a pleasant 72 degrees F. Task completed (Y/N) Task Time (sec) Y 0:05 Follow up questions notes: Notes for syncing the driver side temperature control to the passenger temperature control setting. Task completed (Y/N) Task Time (sec) N Follow up questions notes: Hmmm. What?? (Clicking) that’s not helping. Ahh, there are so many buttons. No I have no idea. There is exactly one button that says sync. I didn’t even realize there is a rear climate setting, based on what I see. I would just reach over and turn the passenger rotary dial. How am I supposed to know there are 4 zones, it’s obvious there are 2. Really nice having multiple climate zones: appealing. I wouldn’t understand without reading the manual.

58

Notes for syncing all 4 temperature zones of the car. Task completed (Y/N) Task Time (sec) Y 0:04 Follow up questions notes: Touchpad: find a person in phonebook Task completed (Y/N) Task Time (sec) Y 1:03 Follow up questions notes: I thought the numbers were printed. What?! Oh that’s cool (figured out letters) (figured out is recognized capital letters) Touchpad: change to radio preset Task completed (Y/N) Task Time (sec) Y 0:04 Follow up questions notes: Great! Touchpad: Select a spot on the map and zoom around using the touchpad. Task completed (Y/N) Task Time (sec) Y 0:34 Follow up questions notes: I don’t want radio. I want a menu. That’s neat, but not what I want (rotary) (Got it, but also wanted to use iPhone touch screen gestures: pinch zoom.) Reasonably intuitive, very responsive, the hardest was figuring out how to change the modes, but once you are in it does exactly what you want. When I use the rotary, I am using the wheel backwards first, I like doing less precise things on touch and get desired outcomes. Have a touch navigation tool is very nice. What bothered me was changing modes, and having the pad change modes depending on context. It wasn’t obvious how to change the function of the touch. I clicked the center to pan, but how do I go back to radio? It’s not doing that. Oh I figured it out, reasonably discoverable. It’s weird to discover the changing context, but not hard to figure it out. Like a less complex version of your phone or what you are already used to. This was much better than talking to it. A common theme I am coming up with, not everything is easy discoverable. With enough fiddling, you get where you need to go, and some people are less forgiving. I am not sure everyone is reading the manual, but I would rather read the manual than more “help” clutter. Clean interface, but discoverability seems tricky. Disappointed it’s not a touch screen. When you have to map from a button to a screen, is really hard problem when you are used to touching the screen. I really

59

think I would recommend the system with the caveat that they would have to spend time reading the manual, there are a lot of buttons but it was not overwhelming. Not sure I prefer the screen over putting my phone in a holder. If the voice was more intuitive I wouldn’t use my phone. I would like to use the system! It was a bit complex but reading about it beforehand would have been very helpful. Limited grammar on the voice, only a couple of times I felt “screw this” I’ll just do it on my phone. Discoverability: the bar is high. Cumbersome? Cumberson feels heavy and clunky, this wasn’t like that Maybe slightly, but may need a better word: parts are not intuitive. Switching contacts with the touchpad was a bit wonky, but Ithink it’s just complication that can be solved by reading the documentation. There are some controls that are not universal and not anything I was familiar with. It gets jobs done but I felt I was floundering, which could have been solved with the manual. Not sure why they didn’t go with touch or easily map able buttons. Seems you’re touching on high use case, how easy is it to do everything people are doing daily. Using lanugauge like “Online destination” doesn’t meaning anything. What about Web”

Participant 9 Notes for first task (Synching phone): Checking phone first to make. Expected rotary to go the other way. These look like two options at first thinking I should go up to click on this, but that didn’t work. I am assuming it’s compatible. Not promising, maybe I need to be in some setting on my phone. Maybe I need pairing mode? Not sure. Now it’s searching. Handsfree? Don’t know what that means. Task completed (Y/N) Task Time (sec) Y 1:46 Follow up questions notes: It wasn’t that positive, wasn’t that easy, lots of warning screens “please check compatibility” didn’t like that, I don’t want to go to the website. Reasonably fast. I don’t like that when it failed it didn’t give me any suggestions on what to do next. I had to work it out on my own. Pretty easy once you figure it out. The confusion about being in ”discovery mode” – how to get in that mode? It could have been smart enough to say something like “no devices found in discovery mode”. I probably want to use Bluetooth for safety, but the setup experience didn’t encourage me to do that. The interface made the experience discouraging. More clear instructions would make it better. It seems like you should turn Bluetooth on and it will connect on its own, but that’s not the case. You need to get it in the right mode, etc. Notes for finding directions to 5038 17th ave NE:

60

It was looking for a radio stations? Directions? Stop. Cancel. Chose help. The whole point on voice is that I have to be in navigation. Part of me wants that to work so I am going to keep trying. “The list is being loaded” The list? I don’t think it’s listening to me anymore. (Left out city, used zipcode) This is not working very well, at this point I would give up. I would use my iphone. I’m frustrated. When I started out I asked for directions, I want directions and not to enter the address… I’m used to more conversational instructions. It seems like I need to use the display a lot, and I probably wouldn’t use this while driving. Task completed (Y/N) Task Time (sec) N Follow up questions notes: Notes for calling a specific person in your phonebook. Task completed (Y/N) Task Time (sec) Y 1:05 Follow up questions notes: Eventually I figured out it’s very sensitive to last names. It probably wants to know the last names. I kind of expected it would pick the only Kristina in my phone. Notes for getting directions to “Pagliacci Pizza”. When I looked at it the distance seemed to change. Task completed (Y/N) Task Time (sec) Y 00:34 Follow up questions notes: Notes for changing Sirius radio to a different channel. Task completed (Y/N) Task Time (sec) Y 00:08 Follow up questions notes: It didn’t interpret my voice well. Behaves different depending on what screen you are on. Unnatural way of speaking. I don’ t think I could use this while driving, but it I were stopped I would just use my phone. I can’t use it while driving because I had to look at the screen a lot, it didn’t give me feedback. When I said “enter address” and made a mistake. I think that every time I make a command I have to look at the screen to verify. Doesn’t seem like a helpful voice interaction. Feedback on what I selected would improve the system. Being able to use It while driving (not having to look at the screen) I would like it to train to my voice. Notes for inputting the driver side climate control settings to a pleasant 72 degrees F.

61

Task completed (Y/N) Task Time (sec) Y 0:09 Follow up questions notes: That’s new to me that you can adjust the driver’s side differently to the rest. Notes for syncing the driver side temperature control to the passenger temperature control setting. I have no idea. Maybe turn it on auto? Oh that turns the whole thing off. I don’t know. I would assume there is only two zones. I would just reach over and turn it. Is there some other control I am not seeing? Task completed (Y/N) Task Time (sec) N Follow up questions notes: I like the colors. Obvious what is hot and cold. It’s cool you can set a specific temp, in my car I have to keep fiddling. I like that there is a temp display. It seems unnecessary that it sets different zones. Too much functionality in terms of syncing and unnecessary to have control over the zones. It could be simpler, have fewer zones. How many zones would you want? Just one. Notes for syncing all 4 temperature zones of the car. Task completed (Y/N) Task Time (sec) Y 0:05 Follow up questions notes: Touchpad: find a person in phonebook That used to have numbers on it. Can I get back? (Delete) I was trying to scroll down, looks like it’s writing letters. Task completed (Y/N) Task Time (sec) Y 1:04 Follow up questions notes: Touchpad: change to radio preset (Forgot to set presets after factory setting) it would be cool if you forgot you were on that station that you are on the preset station. Task completed (Y/N) Task Time (sec) Y 0:28 Follow up questions notes: Touchpad: Select a spot on the map and zoom around using the touchpad. (Still radio buttons on touchpad). Rotary is changing something. (clicking around with rotary) Maybe I need to change a setting? I see there is something called scroll but how do I get there? Oh it changed. Tried to pinch to zoom as I am going kind of slow. I don’t understand how it changed from radio to panning. I would try to figure

62

it out, because I want to know why. I want to change the radio station now, how do I do that? Is it because the radio was on? Hmm I don’t know. Task completed (Y/N) Task Time (sec) Y 1:23 Follow up questions notes: No information about how you would change the mode of the touchpad. The voice commands change depending what screen you are on, but the touchpad seemed to hold on the info of the previous screen. It’s not consistent, unclear why modes changed. I don’t understand why it was on radio. Definitely weird, it has a mind of it’s own. I would use it if I got it into the right mode. I like the position of the hand. And it seemed responsive to drawing letters. It would be useful to use instead of voice. Seems like you are getting an expensive thing that doesn’t work. Spending a lot on an inferior experience. If bought this car I would feel like I got ripped off.

Participant 10 Looks like it should be touch screen. So many things are these days. Notes for first task (Synching phone): DO you push it (referring to rotary knob)? There we go. I’m just going through what you normally do with Bluetooth. At first I was tired so I thought that the car would automatically find my phone but I realized I had to tell the car to find my phone. (Read the pin screen, doesn’t see pin on phone so clicks “no”). So I’ll try again. Oh, there is it. It may have been there earlier but I didn’t read my phone. (Emma checked to see if contacts had synced but PPT said phone said it had because phone said it had.) Task completed (Y/N) Task Time (sec) Y Forgot to get it (task time is stupid, I’m

over it haha). Follow up questions notes: It was actually really easy. Before, when I’ve done this, you have to go into the manual, see what the code is. I’ve never experienced a system that shows you the pin on your phone, without having to go through the manual. Notes for finding directions to 5038 17th ave NE: (When she said “enter address” the system didn’t respond. Tried “enter address” again. It worked. Used zip code. Interrupted by phone call). Oh do I have to do it again? I don’t like this. I am very confused and I don’t know where to tell it to go. (Tries again, says zip code again). I would not go through this a third time. (realistically): I don’t think voice would be the only way to do this. (Do you want to try again): No. I feel like I’m speaking clearly. (third time only says city and state, not zip). Task completed (Y/N) Task Time (sec) Y 4 min 15 sec

63

Follow up questions notes: Notes for calling a specific person in your phonebook: (Used only first name. Was told to use first and last name after saying “correction”. Then got it). That was easy. Task completed (Y/N) Task Time (sec) Y 30 Follow up questions notes: Notes for getting directions to “Pagliacci Pizza”: (Says help, cancels out). Find Pagliacci Pizza. (laughs). (Get’s put in contacts, and clicks out using buttons). Navigation, Pagliacci Pizza. (system doesn’t respond). (Get’s put in contacts again, clicks out). (Realistically): Probably would use my phone. Navigation, find Pagliacci pizza. I don’t know when I say navigation why it takes to my phone. I don’t know what I need to say. I tried find, restaurant..frustrating. (Voice cancelled on her again. Tried to speak when system wasn’t listening. Task completed (Y/N) Task Time (sec) N Follow up questions notes: Notes for changing Sirius radio to a different channel: Siruis change channel. (Pardon, says system) Siruis radio. (Tries to talk): Oh that was too late. Tries to change using channel #. Then uses line #. Task completed (Y/N) Task Time (sec) Y 50 Follow up questions notes: (When filling out survey): The technology is appealing, I just didn’t like it. I don’t know what to say to it. I don’t know if there’s a specific command….I just didn’t connect with it at all. If voice command works, it’s appealing but when it doesn’t…..It’s incredibly annoying. I don’t even use Siri….I’ve just given up on it. I would also read the manual or ask someone. I wouldn’t leave the lot without knowing how to use it. Once I was told I would remember it. I guess if someone was borrowing my car it would be a problem for them. I don’t know how I would improve it because it’s probably fine if I know how to use it. I don’t know…I would maybe on the navigation to try and find, trying to find an address was easy, even though it didn’t understand me. It was frustrating that it didn’t understand me and I had to do it three times but I understood. For finding the place, that wasn’t clear. I see that a few things have line numbers; maybe I needed to use that. I was definitely using the wrong command. Notes for inputting the driver side climate control settings to a pleasant 72 degrees F:

64

(Tries to use rotary knob to change temp. Goes into car on main infotainment screen.) Ooo footwell temperature. I thought I had to use that (referring to rotary knob). Oh I would instinctively to that (referring to temp knob). Task completed (Y/N) Task Time (sec) Y Took a LONG time Follow up questions notes: Notes for syncing the driver side temperature control to the passenger temperature control setting: Had to explain sync. Instinctively, I probably wouldn’t…I probably would go over here (referring to the temp knob). This is just fine for me. Again, I would just look it up. I’m the type of person who, I’ve had several new cars, I’m the kind of person who would ask the salesperson or read the manual. There are many cars I’ve had that don’t have many features but I read the manual and figure them out. If there is a way to sync it, I don’t know and I don’t really care. Task completed (Y/N) Task Time (sec) N Follow up questions notes: Notes for syncing all 4 temperature zones of the car: (Completed while on task 2) Oh there are 4…I didn’t know there were 4. Task completed (Y/N) Task Time (sec) Y 24 Follow up questions notes: (when filling out survey): I would tell them that if you want to get technical with it you can but I never would. Can you use voice to change the temperature? (Emma asks about changing temp and using rotary dial): I thought you pointed to here (rotary knob). I don’t consider that (temp knob) as infotainment. Touchpad: find a person in phonebook Oh, whoa that’s weird. Oh that’s cool, I didn’t know you could do that. (what were you expecting): I don’t know, I thought it would scroll. Oh you can actually start typing the whole name. I don’t know how to undo it. (clicked back button). (Finds emma): Yay I found you but now I don’t know how to select you. Unless I just keep typing you name. (Can you figure out how to call): I have no idea..can I use that (call button?). I don’t know how to use it to actually call you. Task completed (Y/N) Task Time (sec) Y 80 Follow up questions notes: (Can you figure out a way to delete just using the touchpad): got it but didn’t know she did. (What do you think about it): The back

65

gesture was cool..when you said “try to go back” it made sense. And I still can’t figure out how to make it call. Touchpad: change to radio preset (What did you think about that): Easy Task completed (Y/N) Task Time (sec) Y 3 Follow up questions notes: Touchpad: Select a spot on the map and zoom around using the touchpad. (reaches for touchpad and it’s still on radio): It still wants to go to radio. Seeing what turning this knob (rotary knob does). I can see this is changing (points to zoon screen). (do you know how you got into the scrolling mode): I wouldn’t say that this isn’t intuitive, with most things you need to play with it once. I would think that once I’m on the maps screen you would think that it would control that (the map) automatically. Task completed (Y/N) Task Time (sec) Y 53 Follow up questions notes: (When filling out touchpad survey): Hmm what did I like about it? I liked drawing (laughs). (When filling out overall experience survey): I love Audi’s. I’ve always wanted an Audi station wagon. I think that I would need a technical person to use this system, does that mean asking the salesperson before I leave the dealership? (Yea, something like that). Additional information): I’m just dying to know how to get to Pagliacci Pizza. (When told how to do it) I probably didn’t take the time to read each and every one. (When trying to call using the touchpad): Oh I didn’t know I just had to tap it. I didn’t know how long I had to press down for. I thought if I held it down. Something would pop up. Oh you were telling me to get the address, there must be some way (then I show her). Oh that’s cool that you can enter address with the touchpad.

66

Works Cited i What is distracted driving. (n.d.). Retrieved from

http://www.distraction.gov/content/get-the facts/index.html ii What is distracted driving. (n.d.). Retrieved from

http://www.distraction.gov/content/get-the-facts/index.html iii De Ambroggi, L. (2013, March 05). Infotainment in cars to enjoy solid prospects in the future despite subdued growth in 2013. Retrieved from http://www.isuppli.com/automotive-infotainment-and-telematics/marketwatch/pages/infotainment-in-cars-to-enjoy-solid-prospects-in-the-future-despite-subdued-growth-in-2013.aspx iv De Ambroggi, L. (2013, July 18). Automotive infotainment electronics market set for growth in 2012. Retrieved from http://www.isuppli.com/Automotive-Infotainment-and-Telematics/News/Pages/Automotive-Infotainment-Electronics-Market-Set-for-Growth-in-2012.aspx v Global Purchasing Staff. (2013, August 30). Infotainment plays growing role in automotive market. Retrieved from http://http://globalpurchasing.com/consumer-electronics/infotainment-plays-growing-role-automotive-market vi De Ambroggi, L. (2013, July 18). Automotive infotainment electronics market set for growth in 2012. Retrieved from http://www.isuppli.com/Automotive-Infotainment-and-Telematics/News/Pages/Automotive-Infotainment-Electronics-Market-Set-for-Growth-in-2012.aspx vii De Ambroggi, L. (2013, July 18). Automotive infotainment electronics market set for growth in 2012. Retrieved from http://www.isuppli.com/Automotive-Infotainment-and-Telematics/News/Pages/Automotive-Infotainment-Electronics-Market-Set-for-Growth-in-2012.aspx viii De Ambroggi, L. (2013, March 05). Infotainment in cars to enjoy solid prospects in the future despite subdued growth in 2013. Retrieved from http://www.isuppli.com/automotive-infotainment-and-telematics/marketwatch/pages/infotainment-in-cars-to-enjoy-solid-prospects-in-the-future-despite-subdued-growth-in-2013.aspx ix De Ambroggi, L. (2013, July 18). Automotive wireless market to expand by more than 40 percent from 2012 to 2018 . Retrieved from

67

http://www.isuppli.com/Automotive-Infotainment-and-Telematics/News/Pages/Automotive-Wireless-Market-to-Expand-by-More-Than-40-Percent-from-2012-to-2018.aspx x Voice recognition installed in more than half of new cars by 2019. (2013, March 19). Retrieved from http://www.isuppli.com/automotive-infotainment-and-telematics/marketwatch/pages/voice-recognition-installed-in-more-than-half-of-new-cars-by-2019.aspx xi Sauro, J. (2011, February 02). Measuring usability with the system usability scale (sus). Retrieved from http://www.measuringusability.com/sus.php xii Sauro, J. (2011, February 02). Measuring usability with the system usability scale (sus). Retrieved from http://www.measuringusability.com/sus.php xiii Sauro, J. (2011, February 02). Measuring usability with the system usability scale (sus). Retrieved from http://www.measuringusability.com/sus.php xiv De Ambroggi, L. (2013, July 18). Automotive wireless market to expand by more than 40 percent from 2012 to 2018 . Retrieved from http://www.isuppli.com/Automotive-Infotainment-and-Telematics/News/Pages/Automotive-Wireless-Market-to-Expand-by-More-Than-40-Percent-from-2012-to-2018.aspx xv Sauro, J. (2011, February 02). Measuring usability with the system usability scale (sus). Retrieved from http://www.measuringusability.com/sus.php