User Research

User research places people in the center of your design process. It’s all about asking the right questions so that you learn the user’s behaviors, needs, and desires, leading you to build the right solutions. The type of research you will conduct depends on the type of project you’re working on, where you are in the process, your timeline, budget, etc. Follow along as we outline different research methods, risks involved with skipping research, and a brief case study.

Source: When to Use Which User-Experience

Research Methods | nngroup.com

A critical assessment to make prior to conducting user research is determining what type of research you ultimately need. The first decision point is whether you need qualitative or quantitative data. Qualitative research results in non-numerical data and you can complete this type of study with a low number of participants, usually 4-6 people is sufficient. Quantitative data, however, is measured by statistical relevance and therefore you need more participants, usually 20+. Another distinction you should make is the difference between attitudinal and behavioral data. The way people think about a solution and how they actually interact with it are not always aligned. The graph from Nielsen Norman Group charts all user research types into a graph across these distinctions of qualitative versus quantitative and attitudinal versus behavioral. You can align the type of data you need with the corresponding research method. However, you need to be realistic about the constraints of the project as well.

Source: User-Centered DesignProcess Map | Usability.gov

At a high level, everything we discuss in this session revolves around the user centered design process. This slide from usability.gov is a broken-down guide of the steps you’ll take to complete a user centered design process. Overall, this is an accurate visual representation of how we run any of our site builds. Keep in mind, this appears as a very linear process but in reality, there is a decent amount of looping back, around, and forward, especially during the test and refine stage.

When to perform user research methods

Source: User Research Basics | Usability.gov

Guided by the user-centered design process, usability.gov made another great resource to help guide you through choosing the best method for your needs. The table lists common research methods and the content you can measure. For example, first click testing is not a great method to use if you want to know your user better but it will help guide your content, design, and provide quantitative feedback during the test and refine stage.

How do I prioritize what to test?

These are the questions we initially begin to think through before recommending a user research method to a client:

• What is risky not to test before launch?
• What has the greatest business benefit?
• What has the greatest impact on user task success?
• What can you feasibly make changes to?

They help us determine when we should prioritize research on a solution. From there, we suggest putting the user research recommendations into an impact vs. ease of implementation matrix. This matrix can then serve as a conversational tool with your steering committee or project team to align on priorities.

What is the risk of skipping user research?

There is a common misconception that if you build an intranet, people will come. That’t not the case if it isn’t built with the right content users are seeking and structured in an intuitive way. Conducting user research helps alleviate these issues as you launch the solution more informed with the information of what people need and how they use it from conducting user research prior to building the solution. Other potential risks of skipping user research are:

• Solving the wrong problems - your product doesn’t actually align with the goals of the user.
• High risk of rework - why spend the extra time and money rebuilding from scratch when you can build the product to meet the intended goals the first time?
• Additional time investment needed for adoption - again, if you build the wrong thing, users won’t come.
• Reduced likelihood of innovation - if you aren’t talking to your users along the way then you miss the small cookie crumbs they leave that might guide you to a novel and more effective solution.

What if there is no time or budget for user research?

If needed, there are a lot of resources that exist detailing other user research that has been done at a large scale. These are studies and proven successful design patterns already implemented that you can use to guide your decisions in lieu of your own user research. For example, if you’re shopping online typically the filters are on the left side of your screen. In your solution, if you put the filters at the bottom of your screen, your users will likely be confused and may not even see the feature. Creating new design patterns introduces a new learning curve for end users so matching your systems to the real world and other common design patterns will reduce your change management needs dramatically.

One of the best arguments we’ve heard in favor of conducting user research is that if you don’t do it, you’ll likely be spending money later to fix the problems… so technically, you really do have the budget to conduct it in the beginning. These arguments aside, there are a few other suggestions we have in case you really don’t have the budget or resources to conduct the research:

• Complete a heuristic evaluation
• Align your design with other systems
• Connect with an end user. This said, you need to make sure that the representative is actually someone who uses the thing, not just a proxy. Talking to one end user is better than not talking to any.
• Clearly identify the risks involved
• Identify other ways to have touch points with users such as: brown bag lunches, demos along the way, private tours, etc. There is usually something you can do that really doesn’t cost anything. Even if that’s just sitting next to an end user at lunch and casually asking a few questions.

Use Case - Migrating from classic to modern SharePoint online

User research varies so much across projects though there are some patterns. Let’s explore a scenario where you already have an intranet built in SharePoint classic and want to change the structure and move it to modern SharePoint. What research should we conduct to inform this project?

Source: When to Use Which User-Experience Research Methods (nngroup.com)

Returning to the Nielsen Norman Group article about When to use which user-experience research methods, we can explore the different product development stages and example research methods suggested for use at each stage. For this case study, we are working with a product that already exists and therefore, we will focus on the strategize and design stages. This is because we have the opportunity for new directions and new solutions in modern SharePoint Online that we didn’t have before in SharePoint classic. Since we are iterating on a solution, we are also aiming to improve the design that already exists.

Where to begin?

Commonly people will start by looking at the analytics, however, this is a small portion of your story. Just looking at the out-of-the-box analytics will give you the numerical data of how many people viewed a site. It’s difficult to understand user behaviors just via these analytics as it leaves out so much context. Are people frequently visiting this page because they are looking for content that is missing? Are they spending a long time on the page because it is helpful or because it is confusing content?

To better understand end user behaviors, you need to involve the end user through some form of user research which can be user interviews, surveys, or field studies. We recommend conducting user interviews as it affords the opportunity to dive deeper into your questions. As a moderated session, you will be able to continue down an interesting thread of feedback, ask for clarifications, and build relationships with the user. These interviews will help you gather qualitative, attitudinal data.

Beyond those one-on-one interviews, we suggest also completing an organization-wide survey which provides you with the quantitative, attitudinal data. Keep in mind however, that since a survey is attitudinal, your data might be skewed as people tend to only complete surveys if they are feeling strongly which may be frustrated or thrilled. It is more challenging to get neutral reactions in a survey.

Lastly, we recommend conducting usability testing to benchmark the current design, identify pain points, and understand user behavior better. This information will help you build a solution supporting how users are already working.

During the project

We recommend starting anything you build, out of the box or custom, with user need statements. These define who is coming to your solution, what they need to do, and their goals. These user need statements help ensure that all the different users are kept in mind when building. For example, there are content owners but also content consumers who will have very different needs. A manager might consume content differently than an individual consumer. Everyone will interact with the content differently depending on their role. If you run into issues along the way where there is a lot of debate as to what’s the best way to move forward, we suggest A/B testing.

Towards the end

This is where you revisit the user need statements that you created at the beginning and turn them into usability testing tasks. We also suggest conducting another survey so that you have a pathway for feedback from all potential users after launch. Even if the survey is skewed, that feedback is still helpful. Is it just one cranky person or is there something that you didn’t take into account in your solution design that’s a valid point?

We highly suggest conducting user research for anything you build regardless of whether the solution is out-of-the-box or custom. If you don’t, you’re making guesses which will often result in double the work, extra time and expense. As Jakob Nielson stated, “a wonderful interface to the wrong features will fail”.

All Resources

• When to Use Which User-Experience Research Methods (nngroup.com)
• User Interviews: How, When, and Why to Conduct Them (nngroup.com)
• Usability Testing 101 (nngroup.com)
• User Need Statements (nngroup.com)
• Card Sorting: Uncover Users’ Mental Models for Better Information Architecture (nngroup.com)
• Tree Testing to Evaluate Information Architecture Categories (Video) (nngroup.com)
• A/B Testing 101 (Video) (nngroup.com)
• Remote Moderated Usability Tests: How to Do Them (nngroup.com)
• Unmoderated User Tests: How and Why to Do Them (nngroup.com)
• “Customer Surveys are No Substitute for Actually Talking to Customers” (Harvard Business Review)

Do you have any questions for us? Continue the conversation on Twitter with the hashtag #AskSympraxis and mention @SympraxisC.