COIN 2.0 USABILITY TEST
Coin 2.0 is a credit card device whose memory stores the information of multiple cards, so that users only have to carry one credit card with them. The motivation for this project stemmed from the fascinating intersection between hardware and software as software starts to blur the line between the two. We explored how Coin 2.0, a credit card device whose memory stores the information of multiple cards, is set up with its corresponding app and how Coin was used to make a purchase. The goal was to explore potential usability problems that arise between a hardware and software product.
Project Type: Usability Test
Platform: Hardware & Mobile
My Role: Moderator & Qualitative Analysis
Project Duration: 3 Months
Project For: Usability Grad Class
For this project I focused primarily on writing the moderation guide and setting up the tasks that our users would run through. This process included understanding the product and all of the steps from how its set-up works to making a purchase. Additionally I ran a three of the usability studies as well as acted as notetaker for two others. For the final deliverable, I was in charge of processing the qualitative results into digestible pieces of information to create a snapshot view of how users felt about using the product and how well they succeeded at using it.
Coin 2.0 card
Card Reader/Swipe Device
use mobile banking
novice Coin users
Part I: IT Lab (UTA 1.210)
Part 2: Starbucks
Items & devices:
Coin 2.0 + Swipe device
2 Starbucks gift cards
Official Coin manual
Asked to “think aloud” through the tasks [CTA]
Audio recorded during entire test
Timed during IT Lab portion
Asked to take a user satisfaction questionnaire after the test
100% for each task
Time on Task
TOT expected to decrease as participants become more familiar with Coin device
TASK 1 - SIGN IN
“Please open the Coin app and log into an account. The password is touch-based. The password is one long touch followed by five short touches.”
TASK 2 - ADD CARD
“Please add the Starbucks gift card labeled “Card 1” to the device. Please add this card manually without using the swipe device. Label the card “CRD1”. Then use the swipe device to authenticate the card by first selecting CRD1. Feel free to use and consult the provided instruction manual.”
TASK 3 - SYNC CARDS
“Please link CRD1 and CRD2 to the Coin device. Feel free to use and consult the provided instruction manual.”
TASK 4 - SELECT CARD
“Please navigate and select the card named “CRD1” on the Coin device. Feel free to use and consult the provided instruction manual.”
TASK 5 - USE CARD
“Please use the Coin device as a gift card. Please make a purchase that is less than $5.00 using ‘CRD1.’”
Before beginning testing, two pilots were run to ensure task clarity and help the different moderators agree on consistent language.
How do you feel about your overall experience using Coin 2.0?
We included post-task questions as well as post-test questions. Included here are post-test question results, though I am happy to discuss all results if you're interested!
Mean: 21.9 sec
Median: 17.61 sec
Mean: 171. 4 sec
Median: 193.87 sec
Mean: 210.4 sec
Median: 170.8 sec
Time on Task
The most difficult task for users was syncing a card. While thinking aloud and trying to complete the task many users were uncertain about how to proceed because the phone add-on and instruction manual showed different things. It was unsurprising that they faced difficulties and surfaced a major blocker for using the Coin 2.0 product.
Overall users were disappointed, frustrated, and confused by their experience with Coin 2.0. While many saw the value that the product could bring to them, they stated that their initial experience with the hardware would stop them from buying this on their own or recommending it to friends and family. The emojis depict these sentiments. We see that while most wanted to feel neutral about their experience, the more "extreme" responses were negative.
There's always something to learn!
Recruitment is difficult, especially within an academic setting where budget is limited.
No matter how sleek your product looks, or how useful it is, if users cannot figure out the basic set-up they will neber, ever use it.
Pilot! Pilot! Pilot! Our pilots helped us work out wrinkles that would have been embarrassing to discover in front of our participants.
Do not name your product Coin if it is a card.