You have come out of an ideation session with forty ideas. Some are strong, some are filler, and a few are genuinely different. You need to know which five are worth developing, and you need to know before anyone spends money finding out the hard way.
That is what idea screening is for. It is the quick sift that sits in front of the serious investment.
The problem is how most of it is done.
The problem with most idea screening
In most conventional idea screening studies, no single person sees all of your ideas. If you have twenty, each respondent might see five. The allocation is rotated so every idea gets seen the same number of times, randomised, and balanced across the demographics that matter.
It is a perfectly defensible piece of survey design. It keeps the questionnaire short, it stops people getting bored, and the sample sizes hold up.
But it also means your ideas never compete with each other.
“Everyone is scoring things in isolation, and you can end up with a league table built from people who watched different matches,” says Rosie.
What Idea Arena does instead
Everyone sees everything.
The ideas are put into a tournament and worked through in sets, quickly. It is deliberately light, because the ideas at this stage are light. A single sentence is enough to screen. That is the difference between an idea and a concept: not the quality of the thinking, just how much detail is on the card.
Because everything is seen by everyone, the result is a genuine ranking rather than an average of separate scores.
Why a swipe test is not the same thing
Plenty of idea screening tools will work through a list quickly. Most of them run on a swipe. Yes or no, keep or discard, and once something has gone it is gone.
That produces a popularity contest. The safe, familiar, immediately understandable ideas rise, and anything that needs a second look disappears in the first round and never comes back.
“Speed on its own just tells you what people recognise,” says Rosie. “It does not tell you what stayed with them.”
The resurrection round
This is the part of Idea Arena that does not exist anywhere else, and it is the part worth understanding.
When the tournament finishes, people are given the chance to bring something back. One idea, from everything they have seen, that they think should not have gone out.
It sounds small. It changes what you learn. To bring something back, a person has to think across the whole set and remember something specific. What surfaces is not what won, it is what stuck.
“A standard concept test will ask people which idea is the most unique, and they will answer, because you have asked them to,” says Rosie. “The resurrection round captures it rather than asking for it. Somebody has to actively want an idea back, having seen everything else it was up against. That is a much harder test to pass, and it is where genuinely different ideas come through.”
Three measures, not two
Most tournament style tools plot two things: how often an idea wins, and how quickly people choose it. Speed is used as a proxy for instinctive appeal.
Idea Arena combines three. Win rate, speed of decision, and the emotional pull captured in the resurrection round. Those become a single engagement score, and ideas are sorted into tiers on it.
The reason for the third measure is straightforward. Win rate and speed both reward the obvious. Something has to account for the idea that nobody picked first and several people asked for back.
Whether the score actually works
Before Idea Arena went anywhere near a client, it was run against a large concept test that had already been completed, so the results could be compared against something known.
Two things came out of it. The ideas that had performed well in the original test still came out on top, which is what you would want, because a screening tool that reorders everything is not measuring what it claims to. And alongside them, a small number of ideas surfaced that the original process had not brought forward.
“You do not want it to produce completely different answers,” says Rosie. “You want it to produce the same answers plus the ones that got overlooked. That is what the pilot showed, and that is really the whole argument for the resurrection round.”
How long it takes
It depends almost entirely on who you need to speak to. The tournament structure is already built, so the work is the ideas and the audience.
For a mainstream category where the audience is simply people who buy the category, it can turn around overnight, or inside a couple of days. For something more specialist, where the people you need are harder to find, it takes longer. Two weeks is the outside.
It is designed to sit directly behind an ideation session. Come out of the room with the output on Friday, know which ideas are worth developing early the following week.
What you get back
A scorecard, with every idea ranked and sorted into green, amber and red tiers. That part is expected.
What comes with it tends not to be. Alongside the scoring you get two pages of what people actually said, in their own words, against three questions.
Why the winner won.
Why somebody chose to bring an idea back at the resurrection round.
And what they would change about it.
That last one is where development teams usually go first, because it is the closest thing to direction a screening exercise can give you. An idea sitting in amber with several people independently saying they would want it in a smaller format is a very different proposition from an idea sitting in amber that nobody can explain.
“It is a quantitative tool with a light qualitative layer on top,” says Rosie. “You are not just getting a ranking. You are getting it in the words of the people who chose it, which is usually where the next version of the idea comes from.”
It is not a substitute for qualitative research and it is not trying to be. What it does mean is that the numbers arrive with an explanation attached, rather than leaving a development team to work out for themselves why one idea sat twelve places above another.
Where it belongs
Idea Arena is not concept testing and is not trying to be. It sits in front of it.
The job is to get a large, messy pool of early ideas down to a shortlist worth spending money on, without spending much money doing it. Everything after that, the detail, the positioning, the audience profiling, is a different exercise with a different tool.
Get the idea screening right and the development budget goes on the right five ideas. Get it wrong and it goes on whichever five were easiest to understand on a Tuesday afternoon.
Idea Arena is FIS Group’s idea screening tool, built to take a large pool of early ideas down to a shortlist worth developing. To talk through how it would work with your next ideation output, contact Chris Moxon.