The IFComp has just ended so it’s a good time to compare historical IFComp results with scores given to games in IFDB. Are there any differences?
Here are the charts from IFComps 1999–2012, based on IFDB data from November 1st, 2013. I couldn’t find IFComp scores from before 1999 and there wasn’t enough data for 2013 yet.
The games are placed in the chart with horizontal axis showing the IFComp score and vertical axis showing the IFDB score. Hovering the mouse over the dot displays the title. The blue diagonal line is the trendline. If the dot is above the trendline, it means it has been ranked higher in IFDB than in IFComp (in relation to other games in the same comp). Conversely if the dot is below the trendline the game did better in the comp than in IFDB. The distance from the line tells how big the disrepancy is.
Only games with at least 5 votes in IFDB are counted. That includes everything from 2008 on. Older bottom-tier games tend to have fewer votes. The only disqualified highly-ranked game is A New Life with only 4 votes which tied for second place in 2005.
(The charts might not show up if you’re reading this from the RSS feed or Planet IF.)
If we look at how IFComp winners are rated in IFDB compared to games released the same year we notice that the best games aren’t always found from the competition. In fact based on IFDB ratings the comp winner is the best-rated game of the year in only two cases, and more than half the time the comp winner isn’t even the best rated IFComp game in IFDB.
The XYZZY award for the best game of the year goes to an IFComp winner roughly one third of the time, and one third of the time it isn’t even nominated. Only Lost Pig in 2007 hit the jackpot (IFComp winner, highest rated game of the year, XYZZY award).
In the table below the first number column is the ranking among the competition games and the second is the ranking among all games that year. The last column shows if the game was nominated or won the XYZZY award for best game. For example in 1996 The Meteor, The Stone And A Long Glass Of Sherbet won the comp, was nominated for XYZZY and in IFDB is rated the 5th best comp game and the 7th best game of 1996.
|1995||A Change in the Weather||2||9|
|1995||Uncle Zebulon’s Will||1||4|
|1996||The Meteor, The Stone And A Long Glass Of Sherbet||5||7||nom.|
|2002||Another Earth, Another Sky||5||10||nom.|
|2003||Slouching Towards Bedlam||1||2||won|
|2009||Rover’s Day Out||1||5||no|
(Data retrieved 2013-11-21.)
Here are the “best of”s of each year. The number in the “IFDB best of comp” column is that game’s actual placing in IFComp that year.
|Year||Comp winner||IFDB best of comp||IFDB best of year||XYZZY best game|
|1995||A Change in the Weather
Uncle Zebulon’s Will
|Uncle Zebulon’s Will||1||Christminster|
|1996||The Meteor, The Stone And A Long Glass Of Sherbet||Lists and Lists||11||Lists and Lists||So Far|
|1998||Photopia||Photopia||1||Photopia||Spider and Web|
|1999||Winter Wonderland||Exhibition||5||The Mulldoon Legacy||Varicella|
|2000||Kaged||Metamorphoses||2||Metamorphoses||Being Andrew Plotkin|
|2001||All Roads||Heroes||3||First Things First||All Roads|
|2002||Another Earth, Another Sky||Janitor||5||Savoir-Faire||Savoir-Faire|
|2003||Slouching Towards Bedlam||Slouching Towards Bedlam||1||City of Secrets||Slouching Towards Bedlam|
|2004||Luminous Horizon||All Things Devours||3||All Things Devours||Blue Chairs|
|2005||Vespers||Vespers||1||Tomorrow Never Comes||Vespers|
|2006||Floatpoint||The Primrose Path||2||Bronze||The Elysium Enigma|
|2007||Lost Pig||Lost Pig||1||Lost Pig||Lost Pig|
|2009||Rover’s Day Out||Rover’s Day Out||1||Make It Good||Blue Lacuna|
|2011||Taco Fiction||Kerkerkruip||8||Mentula Macanus: Apocolocyntosis||Cryptozookeeper|
|2012||Andromeda Apocalypse||howling dogs||11||Endless, Nameless||Counterfeit Monkey|
(Blue Lacuna is listed as a 2008 game in IFDB, but it was eligible for XYZZYs in 2009. If you count Blue Lacuna as a 2009 game then Violet is the highest rated in 2008.)
What’s obvious here is that people have different criteria for rating games in IFComp and IFDB. Lists and Lists, which teaches how to program in Scheme, is technically impressive but hardly something that people would want to win the IFComp.
There’s also an inherent bias with this kind of scoring where you obviously need to play the game before you can rank it. When you’re participating in the IFComp as a judge you often play the games without knowing anything about them beforehand, or play and rate them all even if you do. Outside the comp you’re likely to not even pick up a genre that doesn’t interest you; if roguelikes are not your thing you’re unlikely to play and rate Kerkerkruip.
Moral of the story? Firstly, even if your game doesn’t rank well in IFComp it might still be highly appreciated in other contexts or as time passes. Secondly, releasing in IFComp is not (and has never been) required for a good game to get the appreciation it deserves.
And finally here are the best IFComp games ever according to IFDB.
|4||Slouching Towards Bedlam||2003||1|
|8||Moments Out of Time||2001||2|
|9||The Best Man||2000||15|