Samsung Harman
Product Research

Discovering Design Improvements of Samsung Harman Car Infotainment by
Need Assessment and Usability Test.

Dedication

I dedicate this research to my deceased friend, Hong-jun. I lost him in a Moroccan dessert in 2007 due to his unfamiliarity with the car UX. I maximized my efforts, knowledge, and expertise to conduct this UX research.


Summary

 
0_infotainment3.jpg
 

Exploring and designing new user experiences in in-vehicle settings prevailed since 2016. Many automotive companies and vendors competitively launched car infotainment systems that interacted with passengers and smartphones. Harman, a Samsung Company, challenged the fierce market too.

We, 4 of UX graduate interns with different backgrounds, took part in Harman's car infotainment system development and conducted extensive UX research to discover design improvements for their working prototype, “Orion” as we named. (This research result is protected by NDA with Samsung Harman.)


Challenge

In order to discover Orion’s usability issues and to improve the product to more user-friendly level, we conducted extensive UX research under the large questions below.

  • What is Orion? How user perceive Orion in comparison to other market products?

  • To which extent does Orion satisfy UX design theories?

  • To which extent does Orion provide user-friendly interaction in real usage?

  • How can we improve and re-design Orion through the UX research data?

 
Research Flow

 
 

First, we fully studied how Orion’s interaction was structured. Then, we compared Orion with competitor market products. Here, we extracted research points for further exploration. 

Then, we qualitatively listened to the voice of potential user group. This data helped us generate quantitative question. The synthesis of qualitative and quantitative was followed. Nielsen's Heuristic Criteria was used to evaluate Orion’s quality. These research data helped us design and conduct the final usability test.

At the end, all the six research outcomes went through iterative validations across each stage. By doing so, we finally suggested design improvement to Samsung Harman’s UX team.

Exploring Solution

Interaction Map Study

We visited Samsung-Harman International to fully study the prototype Orion’s interaction map. We took photos of each screen when interacting with the system.

 
 

Qualitative Research

We conducted in-depth semi-structured interviews for two groups: Samsung Harman UX Team and Potential User Group.

For the first group, we wanted to discover what values the UX team intended to deliver through the infotainment system. For the latter, we wanted to explore users’ general smartphone experiences with car infotainment system, including driving behaviors, goals, motivations, and problems, and phone behaviors in the car.

This approach was to reveal: 1) Frame of user experience with In-vehicle infotainment system; 2) Dimension of current design of in-vehicle system; and 3) Focal usability issues for further investigation.

1) Interview Protocols

  • Two different sets of interview questions

  • Semi-structured, 60 minute for each session

  • Field note-taking & Audio-recording for analysis

  • Thematic Analysis & Affinity Diagram

  • Interviewers’ Attitude

  • Power Relation

  • Avoid preconception & objectification

2) Participant Recruitment

An interview request E-mail was sent to all employees at Samsung Harman International.
4 interviewees were selected after consideration of job titles, demographic characteristics and our meeting schedule. This specifically-targeted recruitment was due to the two main issues.

  • Car Infotainment is a mundane daily UX
    Before conducting interview, we did a pilot test with our interview questions. However, it turned out that car infotainment UX is a mundane daily UX, so that lacking users' attention and interest in general. All the pilot participants did not even sketch how their own car infotainment system looked like.

  • Non-Disclosure Agreement Issue

3) Interview Questions

 

 
4) Data Analysis

The interview data was analyzed with a thematic analysis & affinity diagram strategy, which is widely used in qualitative data analysis (Braun & Clarke, 2006).

We held data interpretation sessions to extract meaningful statements, themes, and theme clusters. We took caution to be true to the interviewee’s words to avoid the potential error of manipulating the data (Lincoln & Guba, 1985).

 5) Findings

(Not presented here due to NDA with Samsung Harman)

 

Persona Creation

We synthesized the interview data and created 3 personas to represent the potential users, by iteratively updating with quantitative data results at later stages.

 
 

 

Comparative Analysis

Upon reflection to Interaction Map Study and Qualitative Research, we framed UX requirements of in-vehicle infotainment systems. Then we listed down Orion’s competitors in three tiers.

 
3_ComAnalysis_2_TierGroup.png
 


1) Research Question

  • Who are the potential competitor to Orion? and Why?

  • What are the similarities and the differences between them in general?

  • How do competitors differ from Orion in terms of functionality, usability, safety, and target population?

  • What aspects can Orion be improved to compete with other products?

 
 

 2) Findings

(Findings are not presented due to NDA with Samsung Harman)

  

Quantitative Research

At this stage, our team conducted a survey to explore users’ phone and driving behaviors, demographics, and infotainment system preferences. Through E-mail recruitment, we received 154 sets of response from the research participants who use car infotainment system on daily basis. Qualtrics was used for analysis.


1) Research Question

We developed survey questions that would: 1) determine the prevalence of the interview findings, and 2) answer what the qualitative research could not reveal.

2) Method

  • Brainstorming Questions
    After defining our research questions, we generated 20 survey questions which falls into the four categories.

    • Information about their car infotainment system and driving habit

    • Phone use in the car

    • Infotainment system use in the car

    • Demographics

  • Pilot Questionnaire & Refining Phrasing
    We decided phrasing, answer formats, and choices each question should contain. We created our initial survey using Qualtrics, and had two friends take this pilot questionnaire to make sure if the questions made sense, and we used their feedback when creating our final questionnaire.

  • Distributing the survey
    We primarily targeted people over the legal driving age, which is 16. We then distributed our surveys through friends, family, and Samsung Harman using social media and E-mail.

  • Analyzing Survey Results
    154 total people completed the survey, and we compared the results to our interview findings. Using Qualtrics data analysis tools, we did cross tabulations and compared graphs for similar questions in order to generate our findings.


    3) Findings

(Detailed findings are not presented due to NDA with Samsung Harman.)

 
 

Heuristic Evaluation

Keeping in mind our findings from interviews, comparative analysis, and survey, we conducted a heuristic evaluation to point out the system’s major flaws and usability issues. Our scope covered not only the overall system interface design, but also, specific functions such as media, phone, and projection mode.


1) Research Question

  • What aspects of the system already function well?

  • What aspects of the system could be improved?

  • How can we apply what we learned about mobile design elements and people’s phone usage habits when evaluating and making recommendations for Orion?

2) Method

Nielsen’s (1994) usability heuristics was used as fundamental principles to conduct this heuristic evaluation on Orion’s interface and functions.  

 
 

 

The scale to rate various usability issues was as follows:

  • 0 = Don’t agree that this is a usability problem

  • 1 = Cosmetic problem

  • 2 = Minor usability problem

  • 3 = Major usability problem; important to fix

  • 4 = Usability catastrophe; imperative to fix

 

3) Findings

(Findings are not presented due to NDA with Samsung Harman)

Onsite Usability Test

At Samsung Harman International, we conducted onsite usability test to assess users’ understanding of the system and to identify any usability issues.

1) Research Question

  • Are users able to navigate the infotainment system quickly and locate settings and music?

  • Are users able to enable projection mode easily?

  • Is the connection process between phone and system intuitive?

These questions are important because if the most frequently used infotainment features are not intuitive, drivers are likely to pay more attention to the infotainment system than the road, resulting in an unnecessarily unsafe driving experience.

2) Method

  • Participant Recruitment
    To honor our non-disclosure agreement, we recruited solely from Samsung Harman’s Novi office.

    • U1 : a 57-year-old male senior quality engineer

    • U2 : a 49-year-old male senior software engineer

    • U3 : a 52-year-old female director of engineering quality and operations

    • U4 : a 57-year-old male security requirements manager

    • U5 : a 40-year-old male HMI senior software engineer

  • Pre-Task Questionnaire
    We created a brief Pre-Task Questionnaire to learn about users’ demographic details and experience level with the infotainment systems, and mobile operating systems since this could influence how easily users would complete the tasks.

  • Task Instructions
    We then created five Task Instructions. From our interview and survey data, we had learned that the most-used infotainment system were music, navigation, and phone calls, so we designed three tasks around these three functions.

  • Consent Form
    For the user to read and sign so they understood the study, their rights, and to obtain written consent before participating.

  • Conducting the onsite Usability Tests
    We printed our pre-test materials and conducted all usability tests at Harman’s Novi office.

 
 

  • Data Logging Spreadsheets
    We identified the steps needed to complete each task and made them the success criteria for each task, putting them into each task’s data logging form.

  • Post-Task Questionnaire
    We quantitatively assessed user impressions of the system after finishing the tasks.

  • Debriefing Protocol
    The moderator and user could have a brief discussion about the user’s thoughts, difficulties, and satisfaction.

  • Analysis
    Each data logger and moderator explained to other team members what happened during their test and what the users said during the debriefing.

    We also aggregated the data from the post-task questionnaires. From these sessions, we found patterns between users, and noted when users made the same errors. Using this information, we extracted key findings which are detailed in the next section.

Findings

(Findings are restricted to be reported to Samsung Harman International only, due to the NDA)

Final Presentation

The final UX research report was presented before 12 UX professionals who developed Orion system.