Usability testing and web analytics are both quantitative UX research methods, but they differ in cost, control, and the kind of insight produced:
- Usability testing is conducted in a controlled or moderated setting and is comparatively expensive, but it lets researchers observe *why* users succeed or fail at specific tasks (e.g., time on task, success rate).
- Web analytics passively records real user behavior in the wild (page views, clicks, feature usage, drop-off points) at low cost, showing *what* users do at scale, but not why they do it.

Difference Between Usability Testing and Web Analytics

Quantitative usability benchmarking measures usability with metrics such as task-completion time and success rate. Researchers can repeat the test to track usability over time or compare performance with competing products. Testing may be conducted in person, remotely with moderation, or remotely without moderation.

University of Michigan - Ann Arbor

Google

Popular types of quantitative user experience (UX) research include:
- Quantitative Usability Testing (Benchmarking)
- Web Analytics (or App Analytics)
- A/B Testing or Multivariate Testing
- Card Sorting
- Tree Testing
- Surveys or Questionnaires
- Clustering Qualitative Comments
- Desirability Studies
- Eyetracking Testing

Popular Types of Quantitative User Experience Research

- Used to detect or prioritize problems and to monitor performance
- Describe what user are doing with the product, where they go, what they click, what features they use, where they come from, and which pages they decide to leave the site or app and etc. 

Web Analytics (App Analytics)

- Used to compare two design options
- Comparing two different live versions of the same UI which are shown to different users to see which version performs best. 
- Great for deciding among different variations of the same design. (Ex - Tracking how many times a certain button was clicked)
- Downside: Some teams fail to run the tests as long as they should, and make risky decisions based on small numbers.

A/B Testing or Multivariate Testing

- Evaluates a hierarchical category structure, or tree, by having users find the locations in the tree where specific tasks can be completed.
- Useful in identifying if an IA structure, labels, and placements agree with people’s expectations.

Tree Testing

- Used to gather information about your users, their attitudes, and behaviors 
- Can produce combination of quantitative and qualitative data — ratings, proportions of answers for each choice in a multiple-choice question, as well as open-ended responses. 

Surveys and Questionnaires

- Used in identifying important themes in qualitative data
- Counting the frequency of instances when a particular theme is mentioned (Ex. Frequency of a user complaint or of a UI problem)
- This technique is less of data-collection methodology, and more of an analysis approach for qualitative data.
- Great way to mine numerical data from large amounts of qualitative information.
-  Downside: Time consuming.



Clustering Qualitative Data

- Used in identifying attributes associated to your product or brand
- Quantify and measure some quality of a product — such as aesthetic appeal, brand strength, tone of voice. (% of respondents describes the design as “Sleek”)


Desirability Studies 

- Used in determining which UI elements are distracting, findable, or discoverable
- Require special equipment that tracks users’ eyes as they move across an interface. 
- Help identify which interface and content elements need to be emphasized or deemphasized, to enable users to reach their goals.
- Very expensive in terms of cost


Eyetracking Testing

Card sorting is a quantitative research method used to determine information-architecture labels and structures. Participants are given content items and asked to group and label them in a way that makes sense to them. Quantitative analysis of the percentage of participants who created similar groupings can help establish which categorization approach would be understandable to most users. These studies can be conducted in person or remotely.

Learn Before

Related

Learn After