It’s February 23rd, 2020. We just spent an hour and a half driving to an industrial zone on the outskirts of Cairo to meet with a couple of our carriers for product testing sessions. We lug our laptops and testing devices out with us and start setting up our recording and streaming system. We spend another painful half-hour setting up the prototype and troubleshooting a bunch of technical issues. Quarter-way into the session, our mobile data hotspot fails and we lose the connection with our team back at the office. A complicated and time-consuming affair, but the outcomes from user research were invaluable to us as a product team. There was no way around it, at least we thought.

Fast forward a month later, and meeting people face to face was not even an option. Our research plans came to a halt. We had to resort to alternative approaches, and luckily, through these approaches, we learned that social distancing doesn’t need to stop us from getting up close and personal with our users. The following is a collection of our thoughts and experiences with user research and conducting it in the COVID-19 era.

Is User Research Even Worth It?

The efficacy of user research is an issue that has divided the product and design community lately. One camp believes that user research and iteration in a simulated environment before launch is meaningless, and that what counts is real-world feedback after the product has launched. The other camp believes that user research is a necessary step in the design process, and that it can and should be done in a controlled environment before launch.

For us at Trella, user research is a necessary step in our product development process, since we face many unknowns when it comes to the nature of the industry and the behavior of our target segment. Some teams may be able to jump into it and launch, given the nature of their product and industry. A team building a productivity or entertainment product for example, where they themselves are their own target audience, could count on releasing an MVP early on and iterating from there. For our team, as far as we currently know, no one has had a previous career in flatbed trailer driving, so it makes sense for us to get into listening mode first.

User Interviews vs. User Testing

User research can take on many forms: surveys, focus groups, interviews, testing, and others. Our focus here is on in-depth qualitative forms like interviews and testing. But between these two, which works best? Generally, pure user interviews provide great insights at any stage in the process, but are especially useful early on in the product cycle; in the exploration and discovery stage. As the product direction begins to take shape, it is much more effective to integrate a testing aspect to the research, where you actually get participants to interact with a prototype. Having a conversation is one thing, but actually getting a true-to-life prototype in front of them reveals much richer insights. There’s a big difference between what users say they’ll do, versus what they actually end up doing. That is where prototyping and testing come in.

Remote User Research: Tips and Techniques

With the COVID-19 situation, given the value of user research for us, we decided to go remote with our research sessions. At first, we were hesitant about the feasibility of getting users to join remote testing sessions unassisted, given that digital literacy levels for those working in trucking and logistics are relatively low. Surprisingly, we faced little difficulty in pulling it off, and we were able to get carriers online with the prototype in front of them in a matter of minutes. Here are three tips from our experience conducting remote user research:

1. Keep It Simple

Not much is needed to set up a remote testing session. No need to get your company to invest in sophisticated testing software, especially given the current economic situation. Any video conferencing application will do just fine. A web link to an interactive prototype also does the job – no need to add friction by requiring the participant to download an additional prototyping app.

2. Comfortable Participant = Truthful Feedback

User testing isn’t an exact reflection of real-world behavior, but you can make sure it’s as close as possible to that. Establish to the participant from the beginning that they’re encouraged to speak their mind, however negative their feedback may be. Also make it clear that “we’re here to test the product, not to test you.” In the interview portion of the session, make sure to transition smoothly between questions, so that it feels more like a friendly conversation than an interrogation. Keep the number of people present in the conference call to a minimum, to avoid placing additional pressure on the participant.

3. Designate an Interviewer + Note Taker

Divide the roles so that there’s one person in charge of asking the questions and guiding the session, while another person focuses on note-taking. Don’t depend on being able to replay the session later since it was recorded; make sure good notes are taken during the interview so they can be summarized and shared with the rest of the team in a concise form.

Whether you’re in product, marketing, operations, or sales, it’s important to always stay close to your customers. Although COVID-19 presented us with an obstacle in that respect, it turned out to be an opportunity for us to introduce research methods and processes that save on time, effort, and resources, while maintaining almost the same degree of effectiveness. Next time we set out to understand our users, we’ll surely be saving ourselves a few headaches.

Ahmed El-Agha, Product Manager at Trella.