Solutions · Product research Which strengths and weaknesses do participants see in your product concepts?
Compare ratings and open-ended reasons for each concept, sort praise, criticism and suggestions by topic, and check every assignment against the original response.
One concept comes out ahead. The open-ended responses say why.
Mean scores show the ranking. Analyzing the reasons behind them takes time again with every test.
The challenge: Separate codes for each concept
If responses are coded separately for each concept, the categories end up different. Strengths and weaknesses are then hard to put side by side.
With deepsight: One category system for all concepts
deepsight builds topics across all concepts and separates praise from criticism within each topic. Strengths and weaknesses line up side by side.
Go to topic analysisThe challenge: Mean scores without reasons
One concept trails on purchase intent. The table doesn’t show what participants dislike about it.
With deepsight: Reasons by rating
Compare which topics participants with low and with high purchase intent mention – with the responses verbatim.
Go to group comparisonThe challenge: Test rounds that are hard to match up
Every revision is followed by a new round. Whether old issues have gone away is then checked by hand.
With deepsight: Every round with the same categories
Analyze each test round with your codebook. You see which criticism fades and which is new.
Apply your codebook
Two concepts, the same questions
Each participant rates one concept and explains the rating in an open-ended response.
Subscription with monthly delivery
“It’s handy that I don’t have to think about it anymore. But I want to be able to pause the subscription at any time, otherwise it would feel like too much of a commitment.”
Select a category to see the matching passage.
Single purchase with reorder reminder
“The reminder is a good idea. But before buying, I’d like to compare the price per pack with other sellers.”
Select a category to see the matching passage.
Illustrative example, 150 participants per concept.
Mentioned often doesn’t automatically mean important
How often a topic is mentioned is an indication. Whether it is decisive for your concept is something you judge in the context of your study. Check categories with only a few mentions against the original responses as well, so individual critical comments don’t get lost.
- Compare the ratings for each concept
- Check strengths and weaknesses of each concept against the original responses
- Collect suggestions for improvement as a category of their own
What your test data can tell you
- What do participants misunderstand or read differently than intended – and what do they find hard to believe?
- Which criticism applies to one concept only, and which to all variants?
- How do participants with low intent to use or buy explain their rating?
- Which suggestions and feature requests come up again and again?
- What gets in the way when people use a prototype or beta version?
- Do target groups, such as existing and new customers, differ in what they praise and criticize?
- Which issues remain after a revision, and which new ones come up?
Good to know: You can also ask the assistant questions like these directly. Its answer separates calculation, evidence and interpretation.
What you start with
The Excel or CSV export from your current survey or testing tool. Open-ended responses, ratings and attributes each sit in their own columns.
From test data to concept comparison
- Prepare the data
Upload the file and add the research question and key terms, such as concept names, to the study context. Filter out empty or content-free responses up front.
Data quality check - Build topics
deepsight suggests categories. Or apply your own codebook – including parent and child categories, strictly or with additional suggested topics; no training required.
Topic analysis - Check and adjust against the original
Read the responses for each category verbatim, correct individual assignments and adjust categories.
Topic refinement - Compare concepts
Put categories, sentiment and ratings side by side for each concept or target group, with significance indicators if you wish.
Group comparison - Share
Share results as a dashboard or prepare them as a report and PowerPoint deck.
Dashboards
Good to know: Names, email addresses or phone numbers in comments are masked by anonymization before the analyses run. If translation is switched on, texts are translated before masking. Anonymization doesn’t catch every detail – check the results before sharing them.
What you hand over to your product team
Good to know: Your tables come from a different analysis program? The slide factory turns them into slides based on your PowerPoint template.
For every strength and every weakness: the responses that back it up.
Your product team discusses what participants actually said, not assumptions.
What matters when you compare concepts
The same categories for all concepts
Analyze the responses for all concepts together instead of building separate topics for each one. Shares are only comparable if they use the same set of categories.
Keep an eye on the base
Shares refer to the analyzed responses in each group; whatever the data quality check filters out doesn’t count toward the base. Multiple mentions are possible. Values with a base below 10 are flagged or hidden in group comparison and dashboards. For small usability tests, it’s best to read every response.
Put differences in context
Significance indicators show whether a concept differs noticeably from the rest of the sample in statistical terms; the test runs from 30 cases per group. With more than two concepts, this is not a pairwise comparison of two concepts. They don’t show why a concept performs differently.
Finalize the code frame before comparing
Categories and assignments are suggestions. Rename, merge, split or remove categories before you compare concepts – every change shifts the shares. Changes are saved as topic versions.
Good to know: You can weight results by a weight column or to target distributions (rim weighting). Weights do not automatically make a sample representative.
Questions about product research
There is no dedicated feature for sequential monadic designs. One option is a file with one row per person and concept, in which the concept rated and its position serve as attributes in the group comparison. Read significance indicators with care in that case: the same people appear in several rows, but the test treats the rows as independent cases. Alternatively, analyze the response columns for each concept with the same categories, for example your own codebook. We will work out with you how best to prepare your data.
You set the priorities, because effort and feasibility aren’t in the responses. Nor can the responses tell you whether a suggestion would improve a rating. As a basis, you see in which groups a suggestion comes up, with the verbatim responses.
Yes, with the same categories in every round. Whether criticism changes because of the revision is something you judge in the context of your test design. Wave comparison is included in every paid plan – see the plan comparison.
deepsight cloud runs on Microsoft Azure in the EU. Data is encrypted in transit (TLS) and at rest (AES-256), and you set retention periods for uploaded datasets per project or for the whole organization. For AI features we use Azure OpenAI; Microsoft does not use your data to train AI models. Support access to your content is only possible with time-limited approval from your admin. DeepL or Google are only used for translation if you choose these services. More under Security.
Related to product research
Analyze your concept test
See the concept comparison in a personal demo, using an example from your product area. Afterwards, you’ll receive test access.