Usability Testing Case Study

UX Designer - May 2017 to July 2017
In Person Usability Testing, Redesign App, and Prototype Redesign.
Before we start

I had the opportunity to be a speaker at the UX Players meetup 6 and share my experience running the usability test. Click here to see the video or here to look at the slides (The talk is in Spanish).


This client has a mobile application (iOS and Android) but is a web application inside a native app (hybrid), it has some odd interactions, confusing flows, and it is slow. The business wants to redesign the App and improve the experience for the new version as a native App.


Identify the strengths and weaknesses of the application by testing basic features/tasks with users and provide opportunities for improvement for the next App version.

The work

I started working with this team at ArkusNexus in May 2017; their client is a company in the USA.
After a couple of meetings with the business and dev team, I asked for the main features/tasks that the users use, and I started reviewing the workflows step by step. After this, I had a meeting with the team and showed the results, and we decided to plan a usability testing with real users to validate our assumptions (never done before in the company).

We planned a 45 min session with the users. We used service to schedule the meetings and we approached the users using our social networks asking for their cooperation. We arranged meetings with five users and proceeded with the usability testing the next days.


Set-up and equipment used to record the sessions.

The users performed four tasks in the App. We recorded each user with two cameras, one pointing the app, and the second to the user to capture their reactions while using the App.


Users testing the application.

The results were analyzed using the KJ Technique (affinity diagram), using yellow post-its for the "errors or problems" and blue post-its for "good comments." The outcome was fantastic; we had real data like the time it took to complete each task, task ease-of-use rating, task success confidence, visual appeal, challenging or frustrating aspects, etc.


Affinity Diagram, the columns are the users, the rows are the activities.

We had a meeting discussing the results and planned what should we do next; to redesign the primary task that has a significant impact on the business (sales).

The redesign goals were:

  • Task completion in 5 minutes or less. The average time to complete this task was 13 minutes with 38 seconds)
  • Minimize the cognitive load of the flow. There was too much information in the form that creates a more cognitive load to the user.
  • Improve the error prevention. Minimize the risk of error in the task.
The redesign

We iterated with low fidelity mockups and finished with an HTML prototype of the new flow; we polished the prototype to test the new design with users to see if we accomplished the redesign goals. Unfortunately, I was moved to a new team at ArkusNexus and wasn't be able to continue with the redesign part.

What I learned and what to improve
  • The schedule and the availability of the user were different and not as I expected. 80% of the users scheduled the test after their work (6 pm), only one user recorded the test in the morning.
  • Put a sign of "please be quiet" outside the testing location. All the test were in a meeting room; there was a lot of noise outside the room, people talking, and even one person entered the room without knocking the door.
  • Have another designer or teammate helping in the sessions (taking notes, etc.).
  • Do not allow cell phones (or put it in vibrator mode); there was a user that received a 15-minute call (death time in the session).