Looney Labs Icehouse Mailing list Archive

[Icehouse] Reliability of the contest results and thoughts

  • From"Jorge Arroyo" <trozo@xxxxxxxxxxxx>
  • DateWed, 20 Feb 2008 12:41:21 +0100
Hi all,

I went ahead and checked how removing some ballots affected the position of my game, and found that by removing just 2 (the ones where it was voted the lowest) it jumped from 7th place to being tied for 2nd place. What this tells me is that we're playing with such a small number of rankings that just one or two people can make the results change drastically. This doesn't seem too good, but I can't really see a way to fix it. Of course, if all the contests before this one had similar number of ballots sent in (except for the last one, in summer) then that says a lot about the reliability of those results too.

Given the current situation, a designer with some friends that sent votes for his game has the potential of taking the competition home unfairly, and this just shouldn't be possible. It's also possible to vote strategically instead of sincerely and be able to affect the results heavily. As an example, if I change my ranking and instead of having Martian12s as first, I put it as last (what I would call voting strategically given that I already knew M12 was the one game that could win), I could force a tie between M12 and WreckTangle. Just one person can do that! Note that I'm not saying this is what has happened in this contest, I'm just pointing out the vulnerability of the system given the low number of votes. For the record, I told my gf not to vote because I didn't want to make it seem like she was voting my game just because it was mine... now I feel stupid.

I can't offer a solution, but I'll offer a suggestion to at least make the competition more useful. Make it a requirement to offer feedback before sending in the votes, or offer feedback along with the votes to be released when the results are. This will make sure, bad ratings are explained and at least the designers can see what people didn't like about their games and how to make them better.

And I'm sorry if I sound like a sore loser. I just think I came up with a nice and original idea for a game that is really different from what is out there and interesting. It probably needs refinement, of course, but again, I don't think it deserved to be ranked 7th...

BTW, I've started a new piecepack competition where instead of games, reviews of games are entered into the contest. We'll see how that turns out, but I'm hoping it will be useful for designers whose games have no feedback. Maybe in the future we could try that with the Icehouse system. Here's the link: http://www.boardgamegeek.com/article/2096186

Cheers,

-Jorge

Current Thread