I devised a way for design researchers under the Watson Customer Engagement portfolio team to measure how “validated” a persona is. I came up with a metric system that would provide consistency across the portfolio. This method was readily adopted and applied on the research team.
Firstly, I created a point system that started off based on arbitrary and subjective numbers to define the value of each research method. Then I held a Mural-based workshop with other design researchers on our team in order to gauge how much they valued each research method.
I adjusted the number of points per method based on feedback from the researchers. I also determined a culminative point system for measuring the validity of each persona:
How it works is each method conducted for a specific persona in mind would add up over a period a time. Generally, as more interviews are conducted, the more confident we are about a persona and the research behind it. However, we want to keep in mind that there are more than just interviews that can support persona validation. This involves a mix of secondary research, surveys, usability tests, and more — as pictured above. We call this “persona cocktails.”
Persona cocktail examples
This is still a work in progress, however, I have created a few examples of potential persona cocktails. What separates different cocktails from each other is based on situational context or what we call “dimensions.” Each of the cocktails provide a recipe for a validated persona, however depending on the circumstance, there are different needs and a different variety of methods. I am also brainstorming other dimensions such as lack of funding or small budget.
Sketch of an idea of persona cocktails
This isn’t a perfect system. I identified some caveats and challenges and did our best to address them. Some thoughts to keep in mind:
To continue to maintain consistency, we need to include not just descriptions to each method but also specific templates and best practices.
There are no ranges for points, they are immutable. But if an interview turned out to be poor or uninsightful, a star rating would be applied next to it. For instance, it would be a 10 point interview with a 3 star rating.
Lastly, I don’t have the answer for it yet, but we are exploring whether points can deteriorate over time, since it is possible for research to be outdated or for a persona to be irrelevant.
I hope that persona cocktails will introduce not only a fun way to measure and compile persona research, but also be able to provide a standard for research and improve consistency and efficiency among our team.