Quantitative Surveys and Panel Recruitment

Quantitative research surveys

Do you use quantitative research methods like surveys? Quantitative research is necessary within the world of UX. This type of research is focused on numbers of responses to provide statistical significance, and consists of popular methods like surveys and analytics within UX.

Quantitative vs qualitative research in UX

Quantitative research results provides hard data in large numbers – it gives you the what. What are users doing? What are the problems with your website?

Qualitative research, on the other hand, is focussed on revealing insights on the why. Why are users behaving in a certain way? Why they are motivated to complete their goal? Qualitative research takes the form of interviews conducted by a professional UX researcher that provides rich data. This data often needs the analysis and interpretation of a UX specialist to uncover the psychological and behaviour insights that are key to making effective changes to your UX designs. 

Quantitative research confirms you have a specific problem.

Qualitative research tells you why you have the problem.

The UX expert tells you what you need to change within your design(s) to resolve the problem. 

Problems UXers have with surveys

A survey conducted with 429 UX professionals by the NNGroup found that many UXers aren’t conducting as many surveys as they’d like. They gave the following reasons:

  • Quantitative research is too expensive
  • Quantitative research is too time-consuming
  • Difficulty recruiting enough participants for large sample sizes
  • Lack of knowledge on the team about how to conduct or analyze quantitative research
  • Lack of knowledge on the team about what quantitative research is, when to use it, or what the methodologies are
  • Lack of understanding of the value of quantitative research
  • Lack of understanding of the value of research in general — not just quantitative research
  • Difficulty interpreting or reporting quantitative research findings

If any of the above resonate with you and your team, then firstly, you’re not alone, and secondly, we would highly recommend getting in touch with us.

We will advise and help wherever you need it. We can also provide a reasonably priced full end to end survey service to take the whole process off your hands.

Panel Recruitment

To recruit users to take part in surveys, you need access to what’s called a panel. Did you know that our sister company, I Need Users, has a huge panel that you can access?

  • Completions from a huge UK wide panel, including Ireland.
  • Quick turnaround. Get hundreds of completions within a matter of days.
  • Niche participants available.
  • Very reasonable pricing.
  • FREE replacements of dropouts.
  • Help linking your survey to our panel (don’t worry, it’s easy!).

I Need Users was created by the founders of Keep It Usable, to provide a higher quality, more reliable participant recruitment service to UX professionals. All of our businesses are focussed on giving you the best possible service.

You might also like:

How to do UX research in a COVID-19 world (plus FREE handbook)

How to recruit users for UX testing (plus FREE recruitment template)

What is User Testing?

How to recruit users for UX testing (plus FREE recruitment template)

User Recruitment

Step 1: Identify who your users are

The first thing you need to do before even thinking about how to recruit is to identify who your users are. You need to know this so that you can pull together the criteria for the types of people you need to include in your interviews. 

Sources to use:

  • Company personas, either from marketing or your UX team
  • Analytics data
  • Customer survey responses
  • Previous quantitative and qualitative research findings
  • Stakeholder knowledge (but ensure they can produce evidence rather than this being just their opinion of who your users are!)

What if I’m using the research to find out who my users are?

For many people, research forms part of the process of creating and validating customer personas. In this case, it’s ok not to have fully formed fancy personas, but you’ll still need at least some idea of who your customers might be so that you don’t waste time speaking with entirely wrong people. 

Step 2: Document your user criteria

Once you’ve done your fact gathering, you’ll need to pull together your user criteria so that it’s easy for you, your team and your recruiter to understand who you’ll need to take part in your research. Don’t worry about this needing to be an all singing, all dancing pdf document. The main purpose of this document is simply to communicate your user criteria. So, this could just be a very basic one page Word document with bullet points listing out the key demographics and behaviours of your user type(s). For other people, this might be a 10 page visual persona document.
Choose whatever works for you!

Step 3: Find users to take part

So, now that you’ve identified the types of people who you need to include in your research, the next step is to find them. There are a few ways you can do this and we’ll start with the least recommended first.

Using internal employees (least recommended)

Using colleagues and internal employees should be your least used method of recruiting users to take part in your research because it comes with a whole heap of bias! But that’s not to say that it doesn’t have it good points and uses. 

Pros

  • It’s free!
  • It’s quick. In a few hours you can have your feedback.
  • It’s useful when you have a very small thing to test and you just want quick feedback on whether people understand design A or design B better.
  • It can be a useful tool for driving company awareness of what UX testing is, future engagement and buy-in. But be careful that you then don’t get stuck in a cycle of internal testing because stakeholders then don’t understand why you can’t just stick with internal employees.

Cons

  • Bias. This is the big problem with this method. These people work at your company, they’re familiar with the brand, values, they’ve probably used the website/app before, they’re more likely to be similar kinds of people because the company have employed them to fit in with the company culture, etc. Sometimes employees can be more positive in their feedback because they don’t want to say anything negative about their employer or they simply really like the company and that’s why they work there, but sometimes they can be more negative, for example if there is a general negative office culture or if the company has recently announced some bad news. Depending on the company size, they may also be on familiar terms with the researcher and this can influence their responses.
  • Employees may be less engaged in the process. We’ve heard of companies forcing employees to take part in research or giving them career incentives, whereas participation in research should be because the person is genuinely interested in taking part and is therefore more highly engaged in the process.
  • Even if they are actual customers, they’re not truly representative. Let’s imagine you work at a fashion company like Topshop. Many of your employees may well be customers, and you use this fact to justify using them in your research. However, your preference should still be to recruit people external to the company to eliminate bias.

How to recruit

  • Stand up, walk to people’s desks and ask them.
  • Put up posters around coffee areas.
  • Send out an email.
  • Setup a stand in a highly frequented area. You could even do your testing at this stand too and make a day of it. 

Recruiting yourself

Recruiting UX participants yourself

When you work within a company, someone will undoubtedly have a list of customers/contacts that you could use to access potential participants for your studies. 

Pros

  • Costs less.
  • You can recruit customers from existing contact lists or online popups.
  • You have full control over the recruitment and quality of participants.

Cons

  • You’ll likely be limited to current customers and lack contact details for non-customers.
  • You’re likely to face red tape within your company and will have to check with legal who you can contact, what for and how you’ll need to do this. It’s likely that your approach and email will need to be agreed by several departments which can cause delays and frustration. You may need to compromise on your requirements simply to get sign off.
  • Takes up more of your time
  • Recruitment can be a very fiddly process. Users may need to call you at inconvenient times or may want to cancel/rearrange their appointment. You’ll need to have availability to deal with them.
  • You’ll need someone to be available during your research to deal with last minute dropouts and replacements, help users with directions if they get lost, questions about car parks and buses, and all the other weird and wonderful things that crop up during recruitment.
  • Bias. There is a bias if you work for your company. There is a bias if you’re the researcher. When people are recruited by an independent party, they may feel more free to be honest in their opinions.
  • There is a bias in only recruiting customers. Customers who say they want to take part in your research may be more likely to be the more engaged positive customers rather than being representative of the average customer. 

Recruiting externally

Fina a user recruitment agency

Ideally, you will recruit people who are external to your company. They will be an actual customer or they will represent your target customer as closely as possible so that your research results more accurately reflect your audience with greater reliability and therefore the impact of the results will be greater. Good recruitment is money well spent.

Using a recruitment panel (most recommended)

An external user recruitment agency with their own panel can be used to recruit both your customers and your representative customers. Using an agency takes advantage of their vast network and frees you up to concentrate on your research. 

Pros

  • Takes the recruitment off your hands and frees you up to concentrate on your research.
  • Access an existing panel of users who have already shown an interest in taking part in research.
  • Useful for recruiting both general and niche users who are harder to find.
  • Gives you easy access to people who aren’t existing customers.
  • Gives you easy access to your competitor’s customers for competitor research.

Cons

  • Costs more.
  • Depending on the company, it can take up to 4 weeks to recruit. But some companies offer much quicker recruitment (iNeedUsers.com).
  • You are relying on a third and most likely a fourth party to accurately screen and ensure attendance of your users (Did you know that most agencies don’t recruit directly themselves? They use a network of freelancers around the country so they themselves have no involvement in the quality or consistency of your recruitment). When people aren’t accurately screened, you may receive users who don’t accurately fit your criteria, who are late or unprepared, who aren’t chatty, who struggle to verbalise their thoughts and opinions, and who are more likely to not attend and to not let you know. Experiences like this end up costing you more in wasted time and can be embarrassing for you when you have stakeholders observing.

We recommend: iNeedUsers.com

iNeedUsers UX user research participant recruitment specialists
  • 100% user attendance for most projects.
  • Free replacement of cancellations and no-shows.
  • Rigorous screening process ensures high quality and good fit participants.
  • Standard 2 week turnaround with options for faster recruitment.
  • Direct recruitment through own panel.
  • Qualitative and quantitative.
  • Trusted by the world’s top brands.

Download your FREE recruitment template

Together with iNeedUsers, we’ve put together a FREE recruitment template to help you create your requirements when you’re recruiting participants.

You might also like:

How to do UX research in a COVID-19 world (plus FREE handbook)

Top 10 major risks of poor user recruitment

Top 10 reasons why good user recruitment is crucial to the success of your UX research

Book Review: UX Strategy by Jaime Levy

UX Strategy by Jaime Levy

UX Strategy: What the heck is it and should you care?

These are the first things you should ask yourself before even thinking about buying this book. For many of you (particularly the old time UXers – you know who you are 😉) you have already been doing UX Strategy for most of your career and simply classing it as part of your work as a UX Professional. It’s all the stuff we do that’s more focussed on the business side. Things like discovery research to explore current user behaviour and needs, competitor analysis, user journey mapping, personas, crafting value propositions, testing them, storyboarding, creating early designs/prototypes and testing them iteratively. Jaime references Eric Reiss’ book ‘The lean startup’ several times throughout her book and the approach detailed in her book is very much in line with lean UX and building MVPs.

In fact, one could argue whether UX Strategy is just another term that’s been created for something that already exists. However, as the definition of UX changes and moves closer to a design discipline, we need a means of people recognising strategy as a key skill. Ironically, as the UX discipline has exploded, key skills like strategy which would ordinarily be part of an experience designers role, are much harder to find. So if UX Strategy needs to be split out into another pillar of UX to increase awareness and skills, then so be it.

So, how does Jaime define UX Strategy?

UX strategy is the process that should be started first, before the design or development of a digital product begins. It’s the vision of a solution that needs to be validated with real potential customers to prove that it’s desired in the marketplace. Although UX design encompasses numerous details such as visual design, content messaging, and how easy it is for a user to accomplish a task, UX strategy is the “Big Picture”. It is the high-level plan to achieve one or more business goals under conditions of uncertainty.

What is UX Strategy

In our opinion, a good UX Designer thinks about strategy as a natural part of their role – it’s what differentiates a UX Designer from a visual designer or a content designer. The UX Designer should always start with the bigger picture before any design takes place. Perhaps Jaime’s limited definition of UX Design is why she felt the need to write a book about UX Strategy – to increase awareness of these important UX skills.

Contents

  • What Is UX Strategy?
  • The Four Tenets of UX Strategy
  • Validating the Value Proposition
  • Conducting Competitive Research
  • Conducting Competitive Analysis
  • Storyboarding Value Innovation
  • Creating Prototypes for Experiments
  • Conducting Guerrilla User Research
  • Designing for Conversion
  • Strategists in the Wild
  • Dénouement

UX Strategy Page Sample

Pros

This is a really interesting read and packed full of useful information and explanations. Jamie uses real examples as much as possible throughout the book and includes templates that you can download through her website too, which is really helpful for you to practise some of the tools in the book.

The book itself is really easy to read, detailed and practical – she breaks everything down into steps to follow. It also includes a lot of visual screenshots so you can really follow the key example she uses throughout the book (an Airbnb for weddings business) and practise the techniques for yourself. This is not a book full of waffle and theory!

You’ll come away with an understanding of the UX process for creating products, and will be able to use the tools to get started on the Strategy process.

Cons

We disagreed with several things within her book and at times too much was left open to interpretation. For example, with regards to user recruitment, she quickly references Nielsen’s 5 user recommendation, but most UX professionals now agree that this figure shouldn’t be used as a one and only rule. The reality is, it’s more complex and careful thought needs to be given to how many users are seen. If you are doing lots of quick iterations you could in theory go lower, or if you have lots of variables and are planning less iterative testing, you may need to go higher.

Jaime also advises guerrilla research in coffee shops, but her recruitment method takes 5-10 days to book people in which goes against many of the benefits of quick coffee shop research. This is a bit confusing – she may as well use a lab. There’s no mention of the hugely positive effects of multiple stakeholders observing the research (this can be a key part of UX Strategy). Her coffee shop approach enables just one client observer to be present, not to mention that the recordings can be incredibly poor (the audio in particular, due to all the background noise). If research is important to you, you will benefit from reading a more detailed book.

At the end of the book there are no references.

The graphics and illustrations look dated and aren’t particularly engaging.

Should you buy it?

If you’re new to UX, lack understanding of how UX fits into business or you want a high level overview of the whole process then this book will be useful for you to read. It would also be useful for anyone building digital products, such as business owners, entrepreneurs and product managers.

Note: All our reviews are independent, not sponsored and based only on our opinion

Guest Interview: Whirlpool’s UX Manager

whirlpool-logo

Brandon Satanek Whirlpool UXFebruary’s Keep It Usable guest interview is with our friend Brandon Satanek. Brandon is the UX Manager of Whirlpool Corporation, a company we’re huge fans of (especially their beautiful KitchenAid products).

 

Welcome to Keep It Usable Brandon!

Foremost, let me say that I appreciate the opportunity to tell you more about our work at Whirlpool Corporation!

What is your role at Whirlpool?

I am the manager of the User Experience Design team at our global headquarters in Michigan. This team works in the Global Consumer Design department where we collaborate with other creative professionals such as industrial and graphic designers, model makers, and color experts. My team focuses on products sold in the North American region.

What kinds of products do you work on?

We work on the original form of “apps” – appliances! We have the pleasure of creating products that make the everyday lives of people a little bit easier. It may be a refrigerator that helps you organize food for an upcoming party or a cherished stand mixer that makes stirring chocolate into your cookie dough less strenuous. And, we also help with those other “apps” too!

Whirlpool-Kitchen-aidPerhaps one of the most dynamic aspects of Whirlpool Corporation is the great diversity of product experiences. We are more than a single appliance company – we are a company of many different brands beyond Whirlpool. In the United States we also have KitchenAid, Maytag, Jenn-Air, Amana and others. In the rest of the world, you will find brands like Bauknecht, Brastemp and Consul. Each of these brands is targeted at a specific customer persona. Helping ensure each of those customers has a unique experience tailored to their needs makes our job all the more interesting.For example, the user interface on products may vary from a simple knob to an intelligent LCD color touchscreen. Within the product interface, design opportunities abound. One day, people on my team may be working on the classic human factors problem of mapping knob controls to cooktop/hob burners. Another day, we may be creating an interactive onscreen wizard to help people remove a wine stain from their favorite silk shirt.

 

What is the role of UX within Whirlpool?

We have the responsibility to ensure our products are useful, usable, and desirable

Our corporate mission is: “Everyone, Passionately Creating Loyal Customers for Life.” We believe our UX role fits perfectly with that goal.

We have the responsibility to ensure our products are useful, usable, and desirable. In some cases, this means taking a leadership role in the product development process. Designers on my team are often called on to drive user interface efforts. They will collaborate with engineers to select the appropriate technology and then develop the basic interaction design. We deliver specifications or other requirements which provide detailed information regarding the behavior of the product.

In other cases, we may act in an internal consulting role. For example, a project team may ask us questions about the ergonomics of a particular solution. Could a consumer reach that last sock in the dryer? When the rack is pulled out, is the force needed acceptable? We can help provide actionable data to guide the design process.

 

Could you share a brief overview of your process?

Our team is very much integrated with Whirlpool Corporation’s official development process which begins with in-depth consumer insights and ends with delivering a finished product to them. At a high level, most UX practitioners would recognize our work as a typical user-centered design process. We just happen to align it to the phases-and-gates process the company follows.

However, some UX practitioners working in software or web design may not be familiar with the requirements of developing for manufacturing. We don’t have the luxury of posting updates to the server after release to address any field issues. Instead, we must make sure we get it right the first time – it’s a great responsibility. Therefore, we spend considerable time in the early development phases to design, test, and iterate our product concepts. By the time tooling and manufacturing has started, we are well onto the next project.

UX practitioners working in software or web design may not be familiar with the requirements of developing for manufacturing.

 

How does user research fit in?

Concept – During the development process, participants try out early prototypes which enable us to iterate quickly through design ideas.

Contextual inquiry – We go into people’s homes and observe how they are using the products to look for future innovation areas. We value continual learning about new behaviors and trends.

Benchmarking – Either very early or very late in the development process, we may benchmark our products against a prior generation or the competition.

Ergonomic evaluations – To validate that consumers will be able to reach certain parts or will have the strength necessary to operate components.

Sensorial evaluations – We want components to be more than just usable, they should also deliver a pleasurable and quality tactile feel during operation.

 

How important is international research?

At a user-level, the types of food stored, cooked, and eaten varies and we must make sure our products fit the lives of the people who use them

International research is very important, but rarely goes by that name. I’m reminded of the joke: “What do they call Chinese food in China? {pause for effect} Food!” You might say I have a hard time calling it international research when the people performing it are local to that particular region.

Whirlpool is a global corporation and we are very fortunate to have design offices all over the world. I have colleagues in Italy, Mexico, Brazil, China and India. Each of these offices tends to be responsible for the projects in that region. Of course, we collaborate and there is overall cross-pollination of best practices.

The importance of this work is underscored by the different consumer needs found worldwide. At a macro-level, there are some products, like dishwashers and clothes dryers, that are not available in certain regions. At a user-level, the types of food stored, cooked, and eaten varies and we must make sure our products fit the lives of the people who use them. Even simple things like ensuring the settings on our clothes washers match the information on the garment tags must be taken into account.

 

Do you have a story of a UX success or failure that stands out as a good lesson for others to learn from?

After seeing people adopt rather uncomfortable postures, an idea was developed to create a platform to raise the products to a more convenient height…it shows how contextual user research can lead to user-centered innovations that directly impact the bottom line.

One classic story involves the invention of the pedestal accessory which can optionally sit under our front-load washing machines. A group of researchers were visiting homes and observed how some participants were struggling to reach clothes inside competitive products. After seeing people adopt rather uncomfortable postures, an idea was developed to create a platform to raise the products to a more convenient height. The team took that idea and developed it further by creating a product that not only raises the height but also includes a drawer to store detergent or other laundry supplies. It’s a great case study because it shows how contextual user research can lead to user-centered innovations that directly impact the bottom line and create a new product category in the industry.

 

That’s a great example of how user research can instigate new product ideas! How much emphasis do you place on age related UX and accessibility?

With an aging population, it is very important that our products meet the needs of a diverse set of consumers. Instead of focusing on any particular disability, our goal is to promote Universal Design principles in our products. We want to maintain the usability and aesthetics for the vast majority of the population, regardless of ability.

A number of years ago, Whirlpool Corporation was awarded a Helen Keller Achievement award for our efforts to help those individuals with vision or hearing impairments. This was, in large part, due to work on a set of laundry products and their use of lighting and sound. For example, as the user pressed a button to cycle through temperature options, they could see a light move from setting to setting, and they could hear a different audio tone (corresponding to a music note) for each setting. In this way, feedback was presented along multiple sensory channels and provided broad usability without requiring special accommodation.

KitchenAid toaster

What are your favourite UX websites and blogs?

As UX continues to grow as a profession, so do the number of corresponding websites/blogs. It is very difficult to pick any favorite as I believe good ideas can come from anywhere. Therefore, I have been turning more-and-more to aggregators that collect the best articles all in one place. On the web, I visit Alltop to find a thorough listing of recent posts. On mobile devices, I use the Flipboard app and subscribe to the “Design” channel for inspiration from many different disciplines.

And for shameless plugs, I believe Lisa is doing a fantastic job on this KeepItUsable blog! In my spare time (not affiliated with Whirlpool Corporation), I have recently started EXPERIENCEdzine to highlight how UX ideas can come from entertaining experiences like Disney parks.

 

Why do you think companies should include UX in their process?

I fundamentally believe it is the right thing to do

First, let me start with the cynical answer. Napoleon said, “There are two levers to set a man in motion, fear and self- interest.” If you’re not investing in UX, your competitor will be. Improving UX directly impacts the bottom line. Whether it is through increased sales, decreased tech support costs, or improved loyalty – simplicity sells.

Second, let me answer it from the heart. I fundamentally believe it is the right thing to do. There are enough terrible things in the world, why add your product or service to that list? We need a bit more delight and a little less drudgery in our lives and UX can help make that happen.

 

What advice would you give to companies who don’t yet have a user-centred approach?

The most common advice I hear is to start small and build gradually from demonstrated product successes. In fact, this is advice I wholeheartedly endorse based on personal experience.

After graduation, I started work for a company that did not yet have a UX team. I was the first person hired with any formal training in the subject. And, I was hired as a temporary contractor – no doubt, they were a little uncertain about the value of the profession. It was a safe risk to take. I was thankful for the opportunity and thankful to be paired with a talented visual designer. Our team redesigned a few smaller software projects and the results were well-received. Soon, our focus expanded and so did the team size.

If there are people reading this who are in that same situation, I would certainly encourage them to take that same small risk. No doubt, there are many professionals and agencies out there very willing to help out in the journey. I’m sure they’ll find, like I have, that the journey will be filled with some of the kindest and best people out there.

Brandon Satanek Whirlpool UX

There are many professionals and agencies out there very willing to help out in the journey…the journey will be filled with some of the kindest and best people out there.

Follow Brandon on Twitter Linkedin EXPERIENCEdzine

Further help and advice

Now you know, like Brandon, that UX is crucial to improve your bottom line, so what should you do now?

Keep It Usable help many different kinds of companies to understand their users. We conduct research with real people and design interfaces using an evidence-based approach; every element has reasoning. Our ergonomist can also work with your hardware team to ensure a seamless user experience.

Get in touch now for no-obligation, friendly help and advice from the award-winning ux agency you can trust.

contact keep it usable
If you’re not investing in UX, your competitor will be!
Would you like to feature on the Keep It Usable blog? Let us know who you are and why people would find your story interesting to read about.