An Algorithm to Predict Success
Raph Koster has a great write-up on computerized movie and song analysis tools. The tools predict commercial success of a given movie spoiler (script summary) or song, respectively. He also discusses one for songs. I would love to have one of these for games. And how cool would that tool be to develop?
December 5th, 2006 at 7:17 pm
I can’t fathom why anyone other than The Suits would want something like this.
Really, even if such a tool were possible, all it would really be doing is judging the test game’s attributes against those of other successful games. So you’re not gauging quality; you’re gauging similarity to other works deemed to have quality. Those are very different things.
December 5th, 2006 at 11:45 pm
The movie tool, as it currently exists, looks to be a fairly standard neural network that uses commercial success to determine relative values for script elements and combinations. Script writing already uses a number of known tools to manipulate the viewer emotionally. It seems like a logical extension to analyze more elements for different criteria.
\”Quality\” is probably too loaded a word to use: I\’d stick with \”commercial success\”.
Given the large data set and slower evolution, movies are a much better candidate for this sort of treatment than games. Games wouldn\’t have sufficient data on most elements, not that anyone could even do a meaningful job of breaking a game proposal down into relevant elements.
As for why anyone would want it, The Suits are already entirely concerned with a game\’s commercial success. Right now, we have the \”creatives\” argue with the \”suits\” in a hand-wavey dance where they attempt to convince them their idea will make tons of money, but nobody actually knows anything, so it\’s this bizarre game where everyone pulls stuff out of their ass and is wrong more than should be statistically possible. I think it\’d actually be easier working with the suits if I had some known criteria.
December 6th, 2006 at 5:59 am
But here’s the thing: Such a tool could only tell you how your game would be a commercial success by being similar to previous commercial successes. It would be useless (and could easily provide false negatives) for evaluating games that might be successful without being similar to previously successful products.
December 6th, 2006 at 7:31 pm
I agree: we’re not even remotely close to the level of sophistication with gameplay design that we are with story design. We’ve got a huge history of writing fiction, yet very few years of video game design. We don’t have the tools or language at this point. We certainly don’t have the data set, or the stable audience.
I think the tool’s fine for movies and songs, because if something’s so radically different that the tool can’t figure it out, it’ll probably be a flop: people run away, scared.
It feels like an open question whether or not there’ll ever be a sufficiently comprehensive language for games, or whether the space of what’s possible to do will just grow faster than we can comprehend it.
Still, if it worked, I’d love to have the tool.