top of page

COIN 2.0 USABILITY TEST

Coin 2.0 is a credit card device whose memory stores the information of multiple cards, so that users only have to carry one credit card with them. The motivation for this project stemmed from the fascinating intersection between hardware and software as software starts to blur the line between the two. We explored how Coin 2.0, a credit card device whose memory stores the information of multiple cards, is set up with its corresponding app and how Coin was used to make a purchase. The goal was to explore potential usability problems that arise between a hardware and software product.

Project Type: Usability Test
Platform: Hardware & Mobile
My Role: Moderator & Qualitative Analysis
Tools: WebEx
Project Duration: 3 Months
Project For: Usability Grad Class
Project Overview
My Contribution

For this project I focused primarily on writing the moderation guide and setting up the tasks that our users would run through. This process included understanding the product and all of the steps from how its set-up works to making a purchase. Additionally I ran a three of the usability studies as well as acted as notetaker for two others. For the final deliverable, I was in charge of processing the qualitative results into digestible pieces of information to create a snapshot view of how users felt about using the product and how well they succeeded at using it.

Equipment

Coin 2.0 card

Card Reader/Swipe Device

Methodology
Participants

07

Participants

5 Females

2 Males

57%

use mobile banking

18-35

years old

100%

iPhone users

100%

novice Coin users

Procedure

Two locations:

  • Part I: IT Lab (UTA 1.210)

  • Part 2: Starbucks

Items & devices:

  • Coin 2.0 + Swipe device

  • iPhone

  • 2 Starbucks gift cards

  • Official Coin manual

Metrics

Participants were:

  • Asked to “think aloud” through the tasks [CTA]

  • Audio recorded during entire test

  • Timed during IT Lab portion

  • Asked to take a user satisfaction questionnaire after the test

  • Completion Rate

    • 100% for each task

  • Time on Task

    • TOT expected to decrease as participants become more familiar with Coin device

  • Subjective Measures

    • Emoji stickers

    • Satisfaction post-task

    • Satisfaction post-study

Tasks

TASK 1 - SIGN IN

“Please open the Coin app and log into an account. The password is touch-based. The password is one long touch followed by five short touches.”

​

TASK 2 - ADD CARD

“Please add the Starbucks gift card labeled “Card 1” to the device. Please add this card manually without using the swipe device. Label the card “CRD1”. Then use the swipe device to authenticate the card by first selecting CRD1. Feel free to use and consult the provided instruction manual.”

​

TASK 3 - SYNC CARDS

“Please link CRD1 and CRD2 to the Coin device. Feel free to use and consult the provided instruction manual.”

​

TASK 4 - SELECT CARD

“Please navigate and select the card named “CRD1” on the Coin device. Feel free to use and consult the provided instruction manual.”

​

TASK 5 - USE CARD

“Please use the Coin device as a gift card. Please make a purchase that is less than $5.00 using ‘CRD1.’”

Before beginning testing, two pilots were run to ensure task clarity and help the different moderators agree on consistent language.

How do you feel about your overall experience using Coin 2.0?

Results

We included post-task questions as well as post-test questions. Included here are post-test question results, though I am happy to discuss all results if you're interested!

Task 1

Mean: 21.9 sec

Median: 17.61 sec

 

Task 2:

Mean: 171. 4 sec

Median: 193.87 sec

 

Task 3:

Mean: 210.4 sec

Median: 170.8 sec

Time on Task

The most difficult task for users was syncing a card. While thinking aloud and trying to complete the task many users were uncertain about how to proceed because the phone add-on and instruction manual showed different things. It was unsurprising that they faced difficulties and surfaced a major blocker for using the Coin 2.0 product. 

Subjective Responses

Overall users were disappointed, frustrated, and confused by their experience with Coin 2.0. While many saw the value that the product could bring to them, they stated that their initial experience with the hardware would stop them from buying this on their own or recommending it to friends and family. The emojis depict these sentiments. We see that while most wanted to feel neutral about their experience, the more "extreme" responses were negative. 

Lessons Learned

There's always something to learn!

​

  1. Recruitment is difficult, especially within an academic setting where budget is limited.

  2. No matter how sleek your product looks, or how useful it is, if users cannot figure out the basic set-up they will neber, ever use it. 

  3. Pilot! Pilot! Pilot! Our pilots helped us work out wrinkles that would have been embarrassing to discover in front of our participants. 

  4. Do not name your product Coin if it is a card.

bottom of page