Quantitative Surveys and Panel Recruitment

Quantitative research surveys

Do you use quantitative research methods like surveys? Quantitative research is necessary within the world of UX. This type of research is focused on numbers of responses to provide statistical significance, and consists of popular methods like surveys and analytics within UX.

Quantitative vs qualitative research in UX

Quantitative research results provides hard data in large numbers – it gives you the what. What are users doing? What are the problems with your website?

Qualitative research, on the other hand, is focussed on revealing insights on the why. Why are users behaving in a certain way? Why they are motivated to complete their goal? Qualitative research takes the form of interviews conducted by a professional UX researcher that provides rich data. This data often needs the analysis and interpretation of a UX specialist to uncover the psychological and behaviour insights that are key to making effective changes to your UX designs. 

Quantitative research confirms you have a specific problem.

Qualitative research tells you why you have the problem.

The UX expert tells you what you need to change within your design(s) to resolve the problem. 

Problems UXers have with surveys

A survey conducted with 429 UX professionals by the NNGroup found that many UXers aren’t conducting as many surveys as they’d like. They gave the following reasons:

  • Quantitative research is too expensive
  • Quantitative research is too time-consuming
  • Difficulty recruiting enough participants for large sample sizes
  • Lack of knowledge on the team about how to conduct or analyze quantitative research
  • Lack of knowledge on the team about what quantitative research is, when to use it, or what the methodologies are
  • Lack of understanding of the value of quantitative research
  • Lack of understanding of the value of research in general — not just quantitative research
  • Difficulty interpreting or reporting quantitative research findings

If any of the above resonate with you and your team, then firstly, you’re not alone, and secondly, we would highly recommend getting in touch with us.

We will advise and help wherever you need it. We can also provide a reasonably priced full end to end survey service to take the whole process off your hands.

Panel Recruitment

To recruit users to take part in surveys, you need access to what’s called a panel. Did you know that our sister company, I Need Users, has a huge panel that you can access?

  • Completions from a huge UK wide panel, including Ireland.
  • Quick turnaround. Get hundreds of completions within a matter of days.
  • Niche participants available.
  • Very reasonable pricing.
  • FREE replacements of dropouts.
  • Help linking your survey to our panel (don’t worry, it’s easy!).

I Need Users was created by the founders of Keep It Usable, to provide a higher quality, more reliable participant recruitment service to UX professionals. All of our businesses are focussed on giving you the best possible service.

You might also like:

How to do UX research in a COVID-19 world (plus FREE handbook)

How to recruit users for UX testing (plus FREE recruitment template)

What is User Testing?

How to recruit users for UX testing (plus FREE recruitment template)

User Recruitment

Step 1: Identify who your users are

The first thing you need to do before even thinking about how to recruit is to identify who your users are. You need to know this so that you can pull together the criteria for the types of people you need to include in your interviews. 

Sources to use:

  • Company personas, either from marketing or your UX team
  • Analytics data
  • Customer survey responses
  • Previous quantitative and qualitative research findings
  • Stakeholder knowledge (but ensure they can produce evidence rather than this being just their opinion of who your users are!)

What if I’m using the research to find out who my users are?

For many people, research forms part of the process of creating and validating customer personas. In this case, it’s ok not to have fully formed fancy personas, but you’ll still need at least some idea of who your customers might be so that you don’t waste time speaking with entirely wrong people. 

Step 2: Document your user criteria

Once you’ve done your fact gathering, you’ll need to pull together your user criteria so that it’s easy for you, your team and your recruiter to understand who you’ll need to take part in your research. Don’t worry about this needing to be an all singing, all dancing pdf document. The main purpose of this document is simply to communicate your user criteria. So, this could just be a very basic one page Word document with bullet points listing out the key demographics and behaviours of your user type(s). For other people, this might be a 10 page visual persona document.
Choose whatever works for you!

Step 3: Find users to take part

So, now that you’ve identified the types of people who you need to include in your research, the next step is to find them. There are a few ways you can do this and we’ll start with the least recommended first.

Using internal employees (least recommended)

Using colleagues and internal employees should be your least used method of recruiting users to take part in your research because it comes with a whole heap of bias! But that’s not to say that it doesn’t have it good points and uses. 

Pros

  • It’s free!
  • It’s quick. In a few hours you can have your feedback.
  • It’s useful when you have a very small thing to test and you just want quick feedback on whether people understand design A or design B better.
  • It can be a useful tool for driving company awareness of what UX testing is, future engagement and buy-in. But be careful that you then don’t get stuck in a cycle of internal testing because stakeholders then don’t understand why you can’t just stick with internal employees.

Cons

  • Bias. This is the big problem with this method. These people work at your company, they’re familiar with the brand, values, they’ve probably used the website/app before, they’re more likely to be similar kinds of people because the company have employed them to fit in with the company culture, etc. Sometimes employees can be more positive in their feedback because they don’t want to say anything negative about their employer or they simply really like the company and that’s why they work there, but sometimes they can be more negative, for example if there is a general negative office culture or if the company has recently announced some bad news. Depending on the company size, they may also be on familiar terms with the researcher and this can influence their responses.
  • Employees may be less engaged in the process. We’ve heard of companies forcing employees to take part in research or giving them career incentives, whereas participation in research should be because the person is genuinely interested in taking part and is therefore more highly engaged in the process.
  • Even if they are actual customers, they’re not truly representative. Let’s imagine you work at a fashion company like Topshop. Many of your employees may well be customers, and you use this fact to justify using them in your research. However, your preference should still be to recruit people external to the company to eliminate bias.

How to recruit

  • Stand up, walk to people’s desks and ask them.
  • Put up posters around coffee areas.
  • Send out an email.
  • Setup a stand in a highly frequented area. You could even do your testing at this stand too and make a day of it. 

Recruiting yourself

Recruiting UX participants yourself

When you work within a company, someone will undoubtedly have a list of customers/contacts that you could use to access potential participants for your studies. 

Pros

  • Costs less.
  • You can recruit customers from existing contact lists or online popups.
  • You have full control over the recruitment and quality of participants.

Cons

  • You’ll likely be limited to current customers and lack contact details for non-customers.
  • You’re likely to face red tape within your company and will have to check with legal who you can contact, what for and how you’ll need to do this. It’s likely that your approach and email will need to be agreed by several departments which can cause delays and frustration. You may need to compromise on your requirements simply to get sign off.
  • Takes up more of your time
  • Recruitment can be a very fiddly process. Users may need to call you at inconvenient times or may want to cancel/rearrange their appointment. You’ll need to have availability to deal with them.
  • You’ll need someone to be available during your research to deal with last minute dropouts and replacements, help users with directions if they get lost, questions about car parks and buses, and all the other weird and wonderful things that crop up during recruitment.
  • Bias. There is a bias if you work for your company. There is a bias if you’re the researcher. When people are recruited by an independent party, they may feel more free to be honest in their opinions.
  • There is a bias in only recruiting customers. Customers who say they want to take part in your research may be more likely to be the more engaged positive customers rather than being representative of the average customer. 

Recruiting externally

Fina a user recruitment agency

Ideally, you will recruit people who are external to your company. They will be an actual customer or they will represent your target customer as closely as possible so that your research results more accurately reflect your audience with greater reliability and therefore the impact of the results will be greater. Good recruitment is money well spent.

Using a recruitment panel (most recommended)

An external user recruitment agency with their own panel can be used to recruit both your customers and your representative customers. Using an agency takes advantage of their vast network and frees you up to concentrate on your research. 

Pros

  • Takes the recruitment off your hands and frees you up to concentrate on your research.
  • Access an existing panel of users who have already shown an interest in taking part in research.
  • Useful for recruiting both general and niche users who are harder to find.
  • Gives you easy access to people who aren’t existing customers.
  • Gives you easy access to your competitor’s customers for competitor research.

Cons

  • Costs more.
  • Depending on the company, it can take up to 4 weeks to recruit. But some companies offer much quicker recruitment (iNeedUsers.com).
  • You are relying on a third and most likely a fourth party to accurately screen and ensure attendance of your users (Did you know that most agencies don’t recruit directly themselves? They use a network of freelancers around the country so they themselves have no involvement in the quality or consistency of your recruitment). When people aren’t accurately screened, you may receive users who don’t accurately fit your criteria, who are late or unprepared, who aren’t chatty, who struggle to verbalise their thoughts and opinions, and who are more likely to not attend and to not let you know. Experiences like this end up costing you more in wasted time and can be embarrassing for you when you have stakeholders observing.

We recommend: iNeedUsers.com

iNeedUsers UX user research participant recruitment specialists
  • 100% user attendance for most projects.
  • Free replacement of cancellations and no-shows.
  • Rigorous screening process ensures high quality and good fit participants.
  • Standard 2 week turnaround with options for faster recruitment.
  • Direct recruitment through own panel.
  • Qualitative and quantitative.
  • Trusted by the world’s top brands.

Download your FREE recruitment template

Together with iNeedUsers, we’ve put together a FREE recruitment template to help you create your requirements when you’re recruiting participants.

You might also like:

How to do UX research in a COVID-19 world (plus FREE handbook)

Top 10 major risks of poor user recruitment

Top 10 reasons why good user recruitment is crucial to the success of your UX research

New UX Book featuring Keep It Usable

There’s a new UX book on the scene and guess what, it features us!

We’ve been getting a bit of a name for ourselves within the UX scene based on the quality of our work and our passion for all things UX. So when Peter Beare and Gavin Allanwood gained approval to create a new UX book they invited us to take part. The book covers an overview of the whole User Experience process, from research with users (our section – chapter 2) through to design and build.

It’s a book that you can easily dip in and out of and is particularly beneficial to those new to UX who need a higher level understanding of the process, tools and techniques that are used to create a good user experience.

User Experience Design: Creating designs that users really love is now available on Amazon for just under £20 – well worth it.

User experience design book - Creating designs users really love By Gavin Allanwood and Peter Beare

“By putting people at the centre of interactive design, user experience (UX) techniques are now right at the heart of digital media design and development. As a designer, you need to create work that will impact positively on everyone who is exposed to it. Whether it s passive and immutable or interactive and dynamic, the success of your design will depend largely on how well the user experience is constructed.

User Experience Design shows how researching and understanding users expectations and motivations can help you develop effective, targeted designs. The authors explore the use of scenarios, personas and prototyping in idea development, and will help you get the most out of the latest tools and techniques to produce interactive designs that users will love.

With practical projects to get you started, and stunning examples from some of today s most innovative studios, this is an essential introduction to modern UXD.”
UX-User-Profiling-Chapter

We particularly like the layout and style of the book as there is an emphasis on imagery and real world case studies that makes the content really easy to consume and particularly engaging.

Below, you’ll see our user experience machine poster. If you’d like an electronic copy of this, you’re more than welcome to download a copy. We also have a few printed copies – if you’d like one just get in touch.

User Experience Machine

Look out for our next book!

We’ve also been invited to appear in another ux book out later this year, so keep your eyes peeled for that one, which will have a more academic slant.