Deep Listening: Charting a Course with a Mental Model Diagram

Summary

Gained an understanding of a new problem space, identified new opportunities for product enhancement, and redefined our roadmap by conducting non-directed user interviews and creating a Mental Model diagram.

The Background

Bluetooth wireless technology is made possible through the collaboration of engineers that represent different member companies and that live and work across many timezones all over the world. These engineers create Bluetooth specifications that defines the technology that developers use to create products that work seamlessly together.

The Bluetooth SIG provides several collaboration tools to assist engineers with tasks like document management, planning, and real-time collaboration.

The Problem

As the newly assigned lead designer, I was tasked with improving these collaboration tools. However, our team didn't have a clear understanding how exactly these tools were used throughout the process of specification development or where to find the best opportunities to make a real impact.

The Goal

We decided to create a mental model diagram to:

Model real user behavior in the specification development process
Map the solutions our tools provide
Identify opportunities by finding gaps between user behavior and our solutions

If you aren't familiar with Mental Model diagrams, they are a type diagram created by researcher Indi Young that models how users think (their reasoning, reactions, and guiding principles) and compares that against the solutions an organization provides to support them. When areas of user thinking don't have solutions that map to them, those areas represent opportunities for the organization to better support their users.

Indi Young quite literally wrote the book on them, so check out her work on Mental Models for more.

An example of a mental model assembled by the Google Analytics team.

The Action

To create the top half of the diagram and model how our users think about developing specifications, we started by identifying interview candidates. We created proto-personas to help guide recruitment and conducted 12 users interviews, ensuring that each proto-persona and their characteristics were represented in our candidate pool.

Our high-level process for developing a mental model.

Messy, but effective: mapping our candidates against our proto-persona dimensions.

Because we wanted to model how our users thought about developing Bluetooth specifications, we didn't want to bias them with leading interview questions. We utilized a non-directed interview style, kicking off the session with a high-level prompt about specification development and then allowing the user to lead the discussion.

Once we completed our interviews, I began analyzing the transcripts. I combed each one for quotes that described the user's thinking. I focused specifically on three core concepts: users' thinking, reactions, and guiding principles.

Highlighting key quotes in transcripts, with a focus on reasoning, reactions, and guiding principles.

After highlighting these quotes, I copied them into a different document, labeled them, and began the process of summarizing them. The summary helps draw out the insight or intent behind the quote and makes it much easier to understand and remember.

Afert all quotes were summarized, I started identifying patterns. I grouped similar quote summaries together into a first level of groupings, and then created another level of groups by matching together similar concepts from the first groupings.

Grouping quote summaries into patterns.

With the analysis complete, I shifted to generating the diagram itself. The first set of quote groups became towers, and groups of towers became larger mental spaces. The resulting diagram turns a mountain of quotes into an affinity diagram that makes high-level thinking concepts clear, but that also invites exploration.

An excerpt of the initial draft of the upper half of the mental model diagram, which models user thinking and behavior.

With our model of user thinking complete, we shifted to understanding how our organizations tools and resources supported our quote towers and spaces. I worked with our product and program managers to create a list of all of the features and solutions we offered to members involved in developing Bluetooth Specifications. Over the course of several workshops, we thoughtfully aligned each solution under the corresponding towers.

These diagrams can be quite large. Ours ended up being about 15 feet long when printed on 24 inch large-format paper, which gave us plenty of room to work side by side when aligning solutions.

Aligning solutions to towers.

The Results

With the diagram completed, we finalized the formatting and dug into the diagram as a team.

An excerpt of the final diagram.

We found that some area's were well supported with solutions, some were weakly supported, and a few weren't supported at all. There were even some cases where we had solutions or features that didn't align to any user thinking at all.

Opportunities (gaps between user behavior/thinking and our solutions) are highlighted in orange.

We created a list of the major gaps the diagram illustrated and then prioritized that list, and we finally had a research-backed picture of key areas to focus on improving for our tools.

The resulting list of priorities and insights were powerful, especially when combined with other types of research. Our team ended up revising major sections of our product roadmap and backlogs to account for the new opportunities identified. The diagram continues to serve as a powerful tool in prioritizing work and determine future user research objectives.