13 Tips for gathering 1-to-10 outcome ratings during Preference interviews.
Conducting a Preference interview is relatively straightforward: Unlike starting with a screen full of blank sticky notes in Discovery interviews, you move through 5 organized areas of enquiry. You’ll ensure success by following these guidelines:
- Read entire outcome statement: You may be tempted to ask, “So what is your importance score for this next one? It’s better to ask each question in the same manner… to drive out as much variability as possible. Read the entire outcome statement: “How important is it minimize the likelihood of box damage from moisture?”
- Remind them of rating definitions: Periodically direct their attention to several of the definitions for the rating numbers. After reading the outcome statement you might say, “… where a “1” is not important at all… a “5” is moderately important… and a “10” is critical.” Reading the entire outcome statement (above tip) and reminding customers of rating definitions may seem unimportant, but they're not: This consistency helps you drive out unwanted variability.
- Deep probing: When customers give you their 1-to-10 ratings for importance and satisfaction, it’s a great time to probe, e.g., “Can you help me understand why this is so important?” This is also a good time to use the “What and Why” probing methodology you learned for Discovery interviews.
- Record notes: Just below the ratings slider bar is a place to record notes. Be sure your Note-taker uses this liberally: You may learn more about an outcome at this point than you did during Discovery interviews.
- Sensitive information: Here’s one caveat to the last point: Sometimes it isn’t appropriate for the Note-taker to record customer comments for all to see. Perhaps the customer is relaying sensitive information, e.g. strategic plans, competitive positions, financial data, etc. If you think this is the case, the Note-taker should relax, while the Observer makes a few private, hand-written notes. Later, these notes can be recorded in the Preference interview Prep Sheet for this interview in the “Confidential Notes” section.
- Handling disagreements: It’s common for customer contacts in the meeting to give different ratings. Maybe the operations person rates importance at 3 and the marketing person rates the same outcome at 9. This is a good time to quietly listen as they debate and make their positions clear to each other: You’ll typically learn a lot if you don't interrupt. If they don’t reconcile their differences, ask the question this way: “From the perspective of your entire company, how important would you rate this?” Their parochial interests will then usually fade away, and you’ll get a rating that accurately reflects their company’s natural decision-making.
- Split the difference: If your customer contacts were debating between ratings of “3” and “9,” you would never suggest recording a “6.” But if they are going back and forth between “7” and “8,” it’s perfectly fine to suggest “7.5.” (That’s why we included half-points in the rating scales.)
- Private voting first: Sometimes one customer in the room will unfairly dominate the discussion… either because of his/her organizational position or personality. If you believe this could be the case, you can quickly go through all 10 outcomes and have each person silently write their ratings on a piece of paper. Then go through the ratings more slowly, this time asking people to volunteer their ratings. If people have already written down their rating, they’ll be more likely to share it, even if it differs from the view of the more outspoken person.
- All decision-influencers: In most cases, your Preference interview will include several people, all from the same company. Your goal is get them to reach a decision that replicates the natural decision-making of the company. But if one or more key functional roles (technical, marketing, sales, operations, safety, etc.) that normally contributes to the company decision-making is missing… your data won’t be reliable. So work hard to get all decision-makers and decision-influencers in the room… or on a web-conference if their physical presence isn’t practical.
- Rapid data entry: You can click the blue “Next” button to move from importance to satisfaction ratings on the 10 outcomes. But if you feel this is slowing you down, you can also simply click the “Tab” key on your keyboard. You can also click directly on any outcome in the list below the 1-to-10 ratings slider bar to go directly to that outcome… without using either the “Next” button or your “Tab” key.
- Compare bubble positions: When you click the “Done” button after rating all 10 outcomes, you’ll see the 10 outcome bubbles arrayed at the top of the screen. This is a good time to check if customers are satisfied with where all the bubbles are positioned relative to each other. If not, click the outcome in the list at the bottom and make any ratings changes there.
- Add new outcome(s): When you get to the last screen for Part 1 (above), there will be a place for customers to add any new outcomes you didn’t ask about in your list of 10. If they suggest any new outcomes, click the blue “Add new outcome” button and record their outcome statement, outcome name, and ratings for importance and satisfaction.
- Adapt for 2 project scopes: What if you're interviewing a paint producer and they say they'd answer your 1-to-10 outcomes ratings differently if the paint was being used by homeowners vs. paint contractors? Ask them which scope is more important. If they say "paint contractors" then ask them to provide ratings for this. When done, ask customers, "Would you change any Importance and Satisfaction ratings if you had homeowners in mind instead? Capture any changes they give you in the "Comments" section of each outcome rating. Then--after the interview--create a new Preference interview for this company for the "homeowners" project scope. There... two interviews for the price of one! (This is called a two-pass interview. See also the BlueHelp article, Strategies for selecting 10 outcomes.)
For more on Preference interviews, see e-Learning Modules 23 and 24 (Preference Interviews A & B), both at www.blueprintingcenter.com > e-Learning. For more details on how AIM clients conduct Preference interviews, see Preference Interview Research Report , multiple project scopes, multiple topics
Keywords: Preference interview, suggestions for Preference interviews, Preference 1-to-10 ratings, read outcome statement, 1-to-10 outcome rating definitions, disagreements on Preference ratings, adding outcomes during Preference interviews