A good idea is not yet a diverse pool of ideas
An AI system can turn a brief idea into a convincing proposal. Whether that also leads to many different ideas and whether people still feel the result is their own are separate questions. These differences are exactly what the CHI 2026 study *Partnering with Generative AI* by Sebastian Maier, Manuel Schneider, and Stefan Feuerriegel examines.
The paper compares different forms of collaboration with the same language model. It therefore gives a more precise view of AI and creativity than a blanket “better” or “worse”: the result also depends on who develops the idea, who makes the decisions, and how many development steps are planned.
What was studied?
The first preregistered randomised study included 486 people after exclusions. They developed new applications for car sensors. The researchers compared five conditions:
- No AI: The idea was developed without support.
- Conventional chatbot: A language model with largely unchanged default behaviour supported the task.
- AI-led development: The model developed the human starting idea further; the person could revise the result.
- Suggestion mode: The AI gave prompts on how the idea could be developed further.
- Question mode: The AI first offered knowledge, alternatives, and analogies, then supported the subsequent development with targeted questions.
One important commonality: participants started with their own idea, developed without AI at first. The study therefore does not simply compare human creativity with fully autonomous idea generation. The question mode was also not a pure question-and-answer conversation without suggestions from the AI.
A second, likewise preregistered study with 640 people tested the pattern in another context: a new product for students in the United Kingdom with a selling price below £40. It added a sixth condition in which AI-led development was iterative. 320 students evaluated the product ideas.
Three metrics, three different statements
Idea quality was captured through human ratings. In the main study, people with experience in the automotive sector assessed originality and usefulness. That is something different from the number of ideas generated or linguistic polish alone.
Diversity referred to how different the ideas were within a group. To measure this, the researchers used text embeddings, i.e. computational representations of semantic similarity in texts. This is a transparent comparison metric, but not a complete assessment of every possible form of creativity.
Perceived ownership, called “ownership” in the paper, was self-reported by participants. It means how much they felt the idea was their own. This does not imply anything about legal copyright, nor does it prove that people later implement the idea with greater commitment.
This separation is crucial for interpretation. A single idea can be rated highly even if it is similar to many others. Conversely, a pool of ideas can be very diverse without every single idea being especially useful.
What the main study actually shows
Both AI-led development and question mode improved rated idea quality compared with the no-AI group, with statistical significance. For suggestion mode and the conventional chatbot, this gain was not demonstrated in the main study.
In the comparison of diversity, question mode outperformed AI-led development. In contrast, no statistically demonstrable difference appeared compared with the conventional chatbot. So the findings do not show that questions win in every respect and against every kind of AI use.
For perceived ownership, question mode scored higher than AI-led development and the conventional chatbot. That does not mean the feeling remained fully at the level of working without AI. On average, the control group was still higher here.
The preregistered directed comparisons were corrected for multiple testing. For the quality gain versus the control group, the standardised effect size was d = 0.36 in question mode and d = 0.55 for AI-led development. These values describe differences relative to the spread of the ratings. They are not a percentage increase in creativity.
Why the second study blocks a simplistic headline
The additional iterative AI-led variant is especially interesting: it too improved rated quality, but it did not show the same homogenisation pattern as the one-off development.
So “If AI develops the idea further, all ideas become the same” would be too crude a summary. The option to change a draft over several steps is itself part of the workflow under study. The paper supports a view of the entire interaction design, not just the labels “human-led” and “AI-led.”
The human-led variants also differed from one another. A system that offers suggestions for improvement is not automatically the same as a system that opens up alternatives and then asks targeted questions. Which steps were combined matters for how transferable the result is.
What this means in practice
When choosing a tool or designing a workshop, different goals can pull in different directions. For a single developed draft, higher rated quality is helpful. For early ideation, it can also matter that the suggestions cover different directions. In collaborative refinement, it is again relevant how strongly participants experience themselves as the authors of their idea.
An example for orientation: ten well-written product proposals are not yet ten different solution approaches. A qualitative comparison would therefore look at both the individual proposals and recurring patterns across the whole pool. That is a practical implication of separating the metrics, not an additional result of the experiment.
The study does not prescribe a universal workflow. It shows that evaluating only the most convincing final text can miss part of the differences.
How current is the finding?
The publication dates from 2026. The tested model was GPT-4.1, model state of 14 April 2025. Publication date and the technology under study are therefore not the same thing.
The results are therefore not a performance comparison of today’s top models. The tasks examined were limited, text-based creative tasks with participants from the general population. This does not allow direct conclusions about professional design teams, multimodal work, long-term learning effects, or the quality of products actually built later.
The still-relevant part is the testable distinction: the quality of an outcome, the diversity of an idea pool, and the felt sense of personal involvement can develop differently. Whether the effects found repeat with a newer model and a different task remains an empirical question.
Source and transparency
Sebastian Maier, Manuel Schneider, and Stefan Feuerriegel: *Partnering with Generative AI: Experimental Evaluation of Human-Led and Model-Led Interaction in Human-AI Co-Creation*. CHI 2026.
The paper is available under CC BY 4.0. This article places the published study in context; no own experiments were conducted.