Overview
This project was about improving an iterated design in the works before launching with the future of audio products. The app is a controller that allows one to install a new feature for a new music listening experience. Stakeholders wanted to see if they design changes improved the usability, understanding, and desire for the app and its new feature.
Role
Test moderator
Research synthesis
Research synthesis
Team
PIC: Allison Burgueno
Project Overview
We wanted to gain some insight into how users unbox, setup, and experience our soundbars and how they would connect it to Alexa to control it with voice.
What's wrong?
Can users discover and install the new feature in the app? Is the feature hidden? How will users rate and feel about the new experience? Will users understand and be successful in completing the setup process? Would users be interested in this feature?
Solutions?
What if we increase marketing and social media hype to advertise and offer this new feature and experience? What if we work with more partners to make the feature more appealing? We can improve process flow and ease of setup, as well as presenting the difference in experience, but discoverability will be up to its media presence.
Goals & Outcomes
We would like to improve the application and setup flow to be more acceptable for the target customers. Additionally, we need to improve the instructions, text, and images, and the value of the new experience after setup.
Users & Audience
The target users are those who are interested in quality, hi-res, and surround sound audio. Additionally, target users may also enjoy live music and studio quality experiences.
Scope
Stakeholders (project managers, engineers, and designers) provided prototype app for use. Because the app is a prototype there were known issues and sensitivities. Moderator may intervene to "wizard of oz" and tell the user their explicit goals. 1 week for in-lab testing, 8-10 tests required. Highly confidential and new software, we used internal employees who must not be involved in any audio marketing or engineering, and has not participated in previous tests.
Process
Define project scope and plan
Because this has been a design in progress for 2 iterations, the project scope did not change. The plan is to conduct multiple iterations of user testing to continue improving the app design and usability.
Define users
To find the users, we send out a local-based company wide email utilizing our research candidate database with a survey I created. Utilizing Qualtrics and following the criterias and exceptions made by stakeholders, I created a survey to ask questions and gauge potential participants music habits, services they use, products they own, etc while staying neutral and unbiased. After comparing with our proposed test session schedule, I contacted the potential participants who best fit the the requirements to participate.
Recruitment and confirmation
We conducted 10 usability tests in our lab, which features a 1 way mirror, and 5 camera system to ethnographically understand the participants thoughts and actions.
Research
We conducted 9 OoBE tests, 7 whom were male, and 2 females. All candidates must own a soundbar and have set up their soundbar themselves. They all had high interest in home theater sound and solutions, as well as own a voice assistant such as Alexa and Google Home.
Usability Test
We begin by asking about their products they own; to gauge why they like and dislike it, how much they paid and why, and how they researched it and bought it. Additionally, we also gauged their interest and knowledge in Hi-res audio and their preferences in music.
They are given a scenario where they own a set of specific headphones and are told about a new feature that the headphones now supports. They are given a phone with a preloaded prototype app, resembling and functioning as the current app, with the new added feature. They are tasked with setting up the feature on the app on the phone, and their feedback in the process and the sound of the new feature. We ask that they voice their thoughts out loud and ask us any questions at any time.
We begin the usability test by letting them start from the front page of the new feature screen. The first few screens are about the introduction and explanation of the feature, as well as providing a demo of the sound. The moderator would time to time ask about their experiences and thoughts about certain aspects at certain "checkpoints" while the participants freely voice and interact with the product and app. After going through the introductions, the participant must go through the process of connecting the accounts with the provided app to ensure the feature works properly. After connecting correctly, they are asked to listen to music and give their ratings.
The biggest issue was in of the feature itself. If the user cannot understand and feel the difference in feature, then is the feature even worth it or valuable, especially after going through a [difficult] process? This can't be solved by UX. This requires audio engineering and marketing. If a feature isn't not wanted, then redesigning the app won't make a difference. However, what we can solve are the obstacles throughout the setup process.
To solve the issue of the app switching, we can introduce clear cut instructions before throwing the user in. We could also provide a reminder notification to return to the app, as we have seen that utilized before in some applications. We could also provide better explanation and benefits of the setup process to provide a better experience for the feature. Users will only want to go through a process such as this if there is a noticeable difference and benefit.
Because this has been a design in progress for 2 iterations, the project scope did not change. The plan is to conduct multiple iterations of user testing to continue improving the app design and usability.
Define users
To find the users, we send out a local-based company wide email utilizing our research candidate database with a survey I created. Utilizing Qualtrics and following the criterias and exceptions made by stakeholders, I created a survey to ask questions and gauge potential participants music habits, services they use, products they own, etc while staying neutral and unbiased. After comparing with our proposed test session schedule, I contacted the potential participants who best fit the the requirements to participate.
Recruitment and confirmation
We conducted 10 usability tests in our lab, which features a 1 way mirror, and 5 camera system to ethnographically understand the participants thoughts and actions.
Research
We conducted 9 OoBE tests, 7 whom were male, and 2 females. All candidates must own a soundbar and have set up their soundbar themselves. They all had high interest in home theater sound and solutions, as well as own a voice assistant such as Alexa and Google Home.
Usability Test
We begin by asking about their products they own; to gauge why they like and dislike it, how much they paid and why, and how they researched it and bought it. Additionally, we also gauged their interest and knowledge in Hi-res audio and their preferences in music.
They are given a scenario where they own a set of specific headphones and are told about a new feature that the headphones now supports. They are given a phone with a preloaded prototype app, resembling and functioning as the current app, with the new added feature. They are tasked with setting up the feature on the app on the phone, and their feedback in the process and the sound of the new feature. We ask that they voice their thoughts out loud and ask us any questions at any time.
We begin the usability test by letting them start from the front page of the new feature screen. The first few screens are about the introduction and explanation of the feature, as well as providing a demo of the sound. The moderator would time to time ask about their experiences and thoughts about certain aspects at certain "checkpoints" while the participants freely voice and interact with the product and app. After going through the introductions, the participant must go through the process of connecting the accounts with the provided app to ensure the feature works properly. After connecting correctly, they are asked to listen to music and give their ratings.
Findings
We found some issues that we didn't think we'd find based on our review before testing. The biggest issue was that the participants didn't have a concrete understanding of the new feature. They couldn't clearly hear the difference and the feedback was very mixed. Participants found that the connection between apps was very confusing and most got lost during the process, which they misunderstood the purpose behind. Switching between the apps used was the number one reason why participants had confusion or usability issues. It wasn't clear that they ahd to switch between apps for the feature to work properly. One aspect of the process which was very crucial was also confusing and physically difficult to perform. This was a make or break, as most participants wanted to give up due to the process taking too long and too difficult to perform. The app itself was also not very helpful as the guide may not operate smoothly, and causes "stress" on the user.Conclusion
The biggest issue was in of the feature itself. If the user cannot understand and feel the difference in feature, then is the feature even worth it or valuable, especially after going through a [difficult] process? This can't be solved by UX. This requires audio engineering and marketing. If a feature isn't not wanted, then redesigning the app won't make a difference. However, what we can solve are the obstacles throughout the setup process.
To solve the issue of the app switching, we can introduce clear cut instructions before throwing the user in. We could also provide a reminder notification to return to the app, as we have seen that utilized before in some applications. We could also provide better explanation and benefits of the setup process to provide a better experience for the feature. Users will only want to go through a process such as this if there is a noticeable difference and benefit.
Takeaways
This is exactly what user testing is for. We didn't expect for the participants to not have a clear understanding of the feature. Because of that, the feature may not fly as high as we all would hope. UX can only do so much, but that is if the user even gets to the process.