Do you use quantitative research methods like surveys? Quantitative research is necessary within the world of UX. This type of research is focused on numbers of responses to provide statistical significance, and consists of popular methods like surveys and analytics within UX.
Quantitative vs qualitative research in UX
Quantitative research results provides hard data in large numbers – it gives you the what. What are users doing? What are the problems with your website?
Qualitative research, on the other hand, is focussed on revealing insights on the why. Why are users behaving in a certain way? Why they are motivated to complete their goal? Qualitative research takes the form of interviews conducted by a professional UX researcher that provides rich data. This data often needs the analysis and interpretation of a UX specialist to uncover the psychological and behaviour insights that are key to making effective changes to your UX designs.
Quantitative research confirms you have a specific problem.
Qualitative research tells you why you have the problem.
The UX expert tells you what you need to change within your design(s) to resolve the problem.
Problems UXers have with surveys
A survey conducted with 429 UX professionals by the NNGroup found that many UXers aren’t conducting as many surveys as they’d like. They gave the following reasons:
Quantitative research is too expensive
Quantitative research is too time-consuming
Difficulty recruiting enough participants for large sample sizes
Lack of knowledge on the team about how to conduct or analyze quantitative research
Lack of knowledge on the team about what quantitative research is, when to use it, or what the methodologies are
Lack of understanding of the value of quantitative research
Lack of understanding of the value of research in general — not just quantitative research
Difficulty interpreting or reporting quantitative research findings
If any of the above resonate with you and your team, then firstly, you’re not alone, and secondly, we would highly recommend getting in touch with us.
We will advise and help wherever you need it. We can also provide a reasonably priced full end to end survey service to take the whole process off your hands.
Panel Recruitment
To recruit users to take part in surveys, you need access to what’s called a panel. Did you know that our sister company, I Need Users, has a huge panel that you can access?
Completions from a huge UK wide panel, including Ireland.
Quick turnaround. Get hundreds of completions within a matter of days.
Niche participants available.
Very reasonable pricing.
FREE replacements of dropouts.
Help linking your survey to our panel (don’t worry, it’s easy!).
I Need Users was created by the founders of Keep It Usable, to provide a higher quality, more reliable participant recruitment service to UX professionals. All of our businesses are focussed on giving you the best possible service.
The first thing you need to do before even thinking about how to recruit is to identify who your users are. You need to know this so that you can pull together the criteria for the types of people you need to include in your interviews.
Sources to use:
Company personas, either from marketing or your UX team
Analytics data
Customer survey responses
Previous quantitative and qualitative research findings
Stakeholder knowledge (but ensure they can produce evidence rather than this being just their opinion of who your users are!)
What if I’m using the research to find out who my users are?
For many people, research forms part of the process of creating and validating customer personas. In this case, it’s ok not to have fully formed fancy personas, but you’ll still need at least some idea of who your customers might be so that you don’t waste time speaking with entirely wrong people.
Step 2: Document your user criteria
Once you’ve done your fact gathering, you’ll need to pull together your user criteria so that it’s easy for you, your team and your recruiter to understand who you’ll need to take part in your research. Don’t worry about this needing to be an all singing, all dancing pdf document. The main purpose of this document is simply to communicate your user criteria. So, this could just be a very basic one page Word document with bullet points listing out the key demographics and behaviours of your user type(s). For other people, this might be a 10 page visual persona document. Choose whatever works for you!
Step 3: Find users to take part
So, now that you’ve identified the types of people who you need to include in your research, the next step is to find them. There are a few ways you can do this and we’ll start with the least recommended first.
Using internal employees (least recommended)
Using colleagues and internal employees should be your least used method of recruiting users to take part in your research because it comes with a whole heap of bias! But that’s not to say that it doesn’t have it good points and uses.
Pros
It’s free!
It’s quick. In a few hours you can have your feedback.
It’s useful when you have a very small thing to test and you just want quick feedback on whether people understand design A or design B better.
It can be a useful tool for driving company awareness of what UX testing is, future engagement and buy-in. But be careful that you then don’t get stuck in a cycle of internal testing because stakeholders then don’t understand why you can’t just stick with internal employees.
Cons
Bias. This is the big problem with this method. These people work at your company, they’re familiar with the brand, values, they’ve probably used the website/app before, they’re more likely to be similar kinds of people because the company have employed them to fit in with the company culture, etc. Sometimes employees can be more positive in their feedback because they don’t want to say anything negative about their employer or they simply really like the company and that’s why they work there, but sometimes they can be more negative, for example if there is a general negative office culture or if the company has recently announced some bad news. Depending on the company size, they may also be on familiar terms with the researcher and this can influence their responses.
Employees may be less engaged in the process. We’ve heard of companies forcing employees to take part in research or giving them career incentives, whereas participation in research should be because the person is genuinely interested in taking part and is therefore more highly engaged in the process.
Even if they are actual customers, they’re not truly representative. Let’s imagine you work at a fashion company like Topshop. Many of your employees may well be customers, and you use this fact to justify using them in your research. However, your preference should still be to recruit people external to the company to eliminate bias.
How to recruit
Stand up, walk to people’s desks and ask them.
Put up posters around coffee areas.
Send out an email.
Setup a stand in a highly frequented area. You could even do your testing at this stand too and make a day of it.
Recruiting yourself
When you work within a company, someone will undoubtedly have a list of customers/contacts that you could use to access potential participants for your studies.
Pros
Costs less.
You can recruit customers from existing contact lists or online popups.
You have full control over the recruitment and quality of participants.
Cons
You’ll likely be limited to current customers and lack contact details for non-customers.
You’re likely to face red tape within your company and will have to check with legal who you can contact, what for and how you’ll need to do this. It’s likely that your approach and email will need to be agreed by several departments which can cause delays and frustration. You may need to compromise on your requirements simply to get sign off.
Takes up more of your time
Recruitment can be a very fiddly process. Users may need to call you at inconvenient times or may want to cancel/rearrange their appointment. You’ll need to have availability to deal with them.
You’ll need someone to be available during your research to deal with last minute dropouts and replacements, help users with directions if they get lost, questions about car parks and buses, and all the other weird and wonderful things that crop up during recruitment.
Bias. There is a bias if you work for your company. There is a bias if you’re the researcher. When people are recruited by an independent party, they may feel more free to be honest in their opinions.
There is a bias in only recruiting customers. Customers who say they want to take part in your research may be more likely to be the more engaged positive customers rather than being representative of the average customer.
Recruiting externally
Ideally, you will recruit people who are external to your company. They will be an actual customer or they will represent your target customer as closely as possible so that your research results more accurately reflect your audience with greater reliability and therefore the impact of the results will be greater. Good recruitment is money well spent.
Using a recruitment panel (most recommended)
An external user recruitment agency with their own panel can be used to recruit both your customers and your representative customers. Using an agency takes advantage of their vast network and frees you up to concentrate on your research.
Pros
Takes the recruitment off your hands and frees you up to concentrate on your research.
Access an existing panel of users who have already shown an interest in taking part in research.
Useful for recruiting both general and niche users who are harder to find.
Gives you easy access to people who aren’t existing customers.
Gives you easy access to your competitor’s customers for competitor research.
Cons
Costs more.
Depending on the company, it can take up to 4 weeks to recruit. But some companies offer much quicker recruitment (iNeedUsers.com).
You are relying on a third and most likely a fourth party to accurately screen and ensure attendance of your users (Did you know that most agencies don’t recruit directly themselves? They use a network of freelancers around the country so they themselves have no involvement in the quality or consistency of your recruitment). When people aren’t accurately screened, you may receive users who don’t accurately fit your criteria, who are late or unprepared, who aren’t chatty, who struggle to verbalise their thoughts and opinions, and who are more likely to not attend and to not let you know. Experiences like this end up costing you more in wasted time and can be embarrassing for you when you have stakeholders observing.
There’s a new UX book on the scene and guess what, it features us!
We’ve been getting a bit of a name for ourselves within the UX scene based on the quality of our work and our passion for all things UX. So when Peter Beare and Gavin Allanwood gained approval to create a new UX book they invited us to take part. The book covers an overview of the whole User Experience process, from research with users (our section – chapter 2) through to design and build.
It’s a book that you can easily dip in and out of and is particularly beneficial to those new to UX who need a higher level understanding of the process, tools and techniques that are used to create a good user experience.
“By putting people at the centre of interactive design, user experience (UX) techniques are now right at the heart of digital media design and development. As a designer, you need to create work that will impact positively on everyone who is exposed to it. Whether it s passive and immutable or interactive and dynamic, the success of your design will depend largely on how well the user experience is constructed.
User Experience Design shows how researching and understanding users expectations and motivations can help you develop effective, targeted designs. The authors explore the use of scenarios, personas and prototyping in idea development, and will help you get the most out of the latest tools and techniques to produce interactive designs that users will love.
With practical projects to get you started, and stunning examples from some of today s most innovative studios, this is an essential introduction to modern UXD.”
We particularly like the layout and style of the book as there is an emphasis on imagery and real world case studies that makes the content really easy to consume and particularly engaging.
Below, you’ll see our user experience machine poster. If you’d like an electronic copy of this, you’re more than welcome to download a copy. We also have a few printed copies – if you’d like one just get in touch.
Look out for our next book!
We’ve also been invited to appear in another ux book out later this year, so keep your eyes peeled for that one, which will have a more academic slant.
