An interview with Zappos

Hi Alex, could you tell us a little about yourself, your team and what you do for Zappos?

I am a customer research professional who applies his Social Psychology background and his passion for research, design, and innovation to the software industry. My professional goal is to help teams create remarkable products and services which make people’s lives easier and more enjoyable.

Currently I am leading UX Research and Web Analytics for the Zappos Family of Companies.  My work includes both hands-on research for all the Zappos online properties as well as mentoring and team development.

In previous positions, I was responsible for research and usability of the products and services for companies like TurboTax (Intuit), State Farm Insurance, and the Active Network.   I have over 15 years of relevant experience – 5 years of academic research and over 10 years of customer research in the software industry.  I received a PhD in Experimental Social Psychology from Clark University.   My areas of research include: defining and measuring emotions, individual differences, usability, and consumer segmentation.

During my academic career, I developed and taught college-level courses in Research Methods, Statistics, and Social Psychology.  I have numerous presentations at professional conferences, several publications in peer-reviewed journals, and several patent applications.

What process do you follow?

The research team is part of the larger UX team.  As a UX team we follow agile, cross-functional process which involves Design Thinking, rapid prototyping, concept testing, iterative and benchmark usability, and a variety of other methods along the creative and development product development journey.

How much research do you carry out with your users?

We do research along the full cycle of: problem definition —> idea generation —> interface design —> product release —> back to problem definition.

What types of research do you do and why?

We combine both qualitative and quantitative research methods.

Qualitative for idea generation and to understand the “why”

• In-depth interviews
• Home visits / Contextual inquiry
• Iterative usability
• Co-creation
• Focus groups

Quantitative to understand the “what” and to quantify opportunities and issues

• Surveys, including Market Segmentation
• Choice-based conjoint studies
• Data mining
• A/B testing
• Benchmark usability

Which UX tools do you and your team use?

• Survey authoring tools
• Sawtooth
• Qualtrics
• Remote usability via WebEx
• Online card sorting
• Optimal Sort
• Analysis tools
• SPSS
• R
• A/B testing – internal platform

How much A/B and MVT testing do you carry out?

Lots.  We do not release anything before A/B testing it.

How do you decide which changes you should research with users first vs which you should simply MVT?

In the idea generation stage and the concept development stage talking to customers and doing iterative research makes sense because it is cheap to start over and make changes until a good design is developed.

If the change has to do with actual customer behavior, we A/B test – actual behavior which ultimately leads to conversion is the best indicator of success.  Asking people what they would or would not do is silly.

Could you give an example of a conversion barrier or challenge you have faced and what steps you took to overcome it?

We discovered some legacy “error” messages we were surfacing to costumers.  Those were ominous-looking and had harsh and non-factual language, e.g. “Fix the following errors” when the customers had not done anything wrong.  We redesigned the messages to be much more helpful and even apologetic on our part.

What do you enjoy most about your work?

Continuing to learn about what makes people tick and how to create new experiences which positively affect the lives of millions of people.  Breaking barriers and silos between Marketing Research, User Research, Web Analytics, and so on.  Mentoring less experienced colleagues.

Could you share one of your top tips with our readers?

Break down barriers and silos between Marketing Research, User Research, Web Analytics, and so on.  Those are based on archaic organizational structures and make no sense from the point of view of the customer.