The Upsiide Idea Screening Methodology

An intro to Upsiide

We all need to innovate - new products, new services, new packages, new variants. And we need to do it in ways that leverage emerging technology - that are faster, cheaper and better. Upsiide's idea screening methodology is upending the classic practice of 'testing' by making it genuinely useful for you and your teams. We do this in three ways:

  1. Making the process of setting up studies, sourcing respondents and analyzing the results simple and fast
  2. Reinventing the respondent interface to look not like a survey, but like the social media platforms that they enjoy
  3. Using sophisticated modeling to produce results that are predictive of in-market potential

Told you; we're a bit different.

Upsiide - Dig Insights' innovation insights platform - was developed by the agency's research and data science pros, which explains why it's so unique. It's on the Upsiide platform that our clients conduct Idea Screening. In order to validate our methodology for Idea Screening, researchers at Dig conducted hundreds of studies across a variety of industries, with over 8 million assessments collected.

Explaining the Idea Screening Outputs

When you run an Idea Screen on Upsiide, there are three main data points to explore in your results.

Interest Score:

This is the proportion of people who liked an idea. The Interest Score is a percentage score.

Trade-Off Score:

When respondents like two ideas, they are asked to trade-off between them, picking a favourite. The Trade-Off Score is also a percentage score.

Upsiide Score:

This is a composite metric - so, an absolute score - that's been calibrated to predict sales in-market. The Upsiide Score is based on a combination of Interest Score and Trade-Off Score. Though it might sound simple, we fine-tuned the relative weights of the Interest and Trade-Off Scores (using the Hierarchical Bayesian (HB) Linear Model) and consistently validated them with in-market realities. This resulted in an optimized Upsiide Score, a brand new metric that will help you predict sales volumes.

Proof of Upsiide Scores Predictive Power

When we compare Upsiide Scores to market sales data, across a range of industries, we can see how robust the results are. We've compared Upsiide Score to sales in a range of studies and see correlations between 0.58 to 0.99, though typically the correlations sit around the 0.80-0.90 mark.

Novelty Category (Correlation of 0.72) 

novelty category

Seasonal Goods Category (Correlation of 0.99)    

seasonal goods category

Snacking Category (Correlation of 0.80) 

snacking category

     Bakery & Bread Category (Correlation of 0.82)

bakery and bread category

New Snacking Category (Correlation of 0.58)

new snacking category, limited distribution

Small Appliance Category (Correlation of 0.71)  

small appliance category

If you have any further questions about in-market correlations, please don't hesitate to get in touch with one of our Upsiide Experts.

Sophisticated Data Visualization is Built-In

If Upsiide produced nothing more than an Interest Score, Trade-Off Score, and an Upsiide Score, it would be a useful idea screening tool. But we weren't satisfied with "useful". One of the benefits of using a tool built by Dig Insights, an insights & strategy consultancy, is that we understand how to model and visualize data. We offer four different results views, two of which are particularly compelling:

Quadrant Chart

A quadrant chart allows you to compare Interest Score vs. Trade-Off Score. The chart highlights:

  • Ideas that perform well on both measures (clear winners).
  • Ideas that perform poorly on both measures (clear losers).
  • Ideas that not many people like, but perform very well in trade-off (niche ideas). These ideas have the potential to secure a small but passionate user base, and in established categories, they often represent new opportunities for growth.
  • Ideas that are compelling, but not necessarily appealing enough to perform well in-market (compelling but vulnerable).

In this way, Upsiide's idea screening isn't simply a "kills new thinking" methodology. Because of the quadrant chart, we also see ideas that have strong Interest Scores but weak Trade-Off Scores. These ideas often have a strong central idea, but there is a need to revisit some elements (e.g. the insights or the benefit mix).


Network Map Chart

A network map reveals the structure that underlies your ideas. When doing an idea screen, if you include in-market offerings in your test, you see how your brand's innovations fit within the established market.

For example, we recently ran an idea screening study on political policies in the USA. The network map revealed a divided political environment, with clear Democrat and Republican platforms. The most Republican policy is building the border wall. The most Democrat policy is the green new deal. We also revealed common policy areas that were popular with both parties. Most of these focused on controlling healcare costs. As Biden seeks to reach out to Republicans, he would be well-served to focus on healthcare costs.

Imagine if you had the same strategic view of how your innovation ideas relate to each other, or established market players?


Explaining the Respondent Experience

So, you know what you'll get from the results of an Idea Screen on Upsiide. But how do the respondents answer questions about your ideas?

We've leveraged the intuitive swiping methodology that's pervasive on mobile (Tinder, anyone?) and used it to gauge respondents initial interest and engagement with statements, images, logos, and more. Respondents are shown each 'idea' individually; if they like the idea, they swipe right, if they don't like the idea, they swipe left.



Here's where it get's interesting. Once a respondent has liked (or swiped right on) two ideas, they'll be faced with a head-to-head or trade-off scenario: which of the two ideas do they prefer?

This forced choice methodology is crucial to determining the Upsiide Score, as well as differentiating between ideas that are appealing (but not compelling) vs. ideas that are true winners. Not to mention it makes for a thoroughly engaging and gamified respondent experience.

If you're keen to learn more about Upsiide, and how you can leverage the platform within your innovation process, let's have a chat.