Part 4 of the design process

Testing on real users

With an initial prototype finished its time to test it on real users for authentic feedback

Find out more

The test plan


How, why and what I was testing

To decide whether my initial design was an improvement on the original I conceived an A/B test of sorts to test both side by side. The user would try to complete a task (finding a particular video) on the current Hudl app, and then try the same task on the prototype or vice versa. With both versions having exactly the same data and videos. During and after the test I recorded an array of quantitative and qualitative feedback to compare the two.

Check out the first prototype

Data collected during testing


Time

How long it take to complete each task is a good indicator of success.

Errors

How many mistakes the user makes informs us whether it is an intuitive system or not.

Ease of use

A qualitative experience, how easy did they find each version.

Preference

Another qualitative factor, which do they enjoy the most.

User 1

Had used the Hudl Technique several times before

Didn't use the dropdown to filter videos on the original app she simply scrolled through. This led to her completion time being 43% quicker on the new prototype. She found both versions easy to use but preferred the ability to search in the new prototype.

User 2

Had never used Hudl Technique before

Used the search and tab functionality in the new app but didn't try and filter or use the folders. Completion time was 400% faster on the new prototype due to awareness and use of the filtering tabs and search functionality. Preferred the prototype for this same reason.

User 3

Had never used Hudl Technique before

Completion time was 200% faster due to use of the tabs for filtering the videos. Explored the sorting option on the new prototype but didn't try to use. Preferred the new prototype but didn't see the need or use of the folder system.

What users said about it


"The filter and sorting ability wasn't very good but the tabs were"

Nicola Anderton

"The tabs are a definite improvement to the drop down selector, seemed a lot quicker to get where I wanted"

Dean McBride

"Search functionality makes it a lot quicker to find videos"

Zoe Brown

Conclusions


Folder functionality

Not a single user looked into the folder functionality added in the prototype, leaving me to believe they are superfluous feature.

Sorting videos

Reaction to the filter and sort section was generally negative and wasn't used.

Tab and search functionality

Due its obviousness compared to the dropdown this was used successfully by all users and increased task completion time significantly when it was.

Up next

Part 5 of the design process: Iterate

Using the conclusions of testing to improve the app through design iteration